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15.  Supplementary  Notes 

This  effort  was  performed  in  cooperation  with  FAA  Technical  Center  (ACW-200B) 
i  personnel  and  project  personnel  from  Raytheon  Service  Company  in  Pleasantville, 

New  Jersey. 

16^  Abstract 

This  report  summarizes  Operational  Test  and  Evaluation  (OT&E)  activities  for  the  New 
Generation  Runway  Visual  Range  (RVR)  system.  Testing  consisted  of  an  initial  OT&E,  seven 
individual  retests  and  several  specialized  tests.  DOT/FAA/CT-TN92/37  provides  results  of  the 
initial  OT&E  conducted  in  March  1991  This  document  summarizes  results  of  seven  retests  as  well 
as  specialized  tests  conducted  from  August  1992  through  June  1994. 

The  purpose  and  intent  of  OT&E  was  to  verify  RVR  National  Airspace  Requirements  (NAS) 
and  to  verify  the  operational  effectiveness  and  suitability  of  the  RVR  within  the  NAS  environment 

At  the  completion  of  the  retest  and  specialized  test  efforts  results  indicated  that  the  most 
significant  sensor  and  system  problems  had  been  resolved  via  permanent  design  changes  as 
well  as  interim  "work-arounds.”  It  was  recommended  that  the  RVR  system  be  dq)loyed  nationally 
under  the  following  conditions: 

a.  Additional  data  be  obtained  indicating  RVR  performance  during  Category  mb  visibility;  and 

b.  Problems  currently  having  interim  woik-around  solutions  be  resolved  with  permanent  corrections. 

This  volume  contains  a  summary  of  the  retest  and  specialized  tests  performed  during 
the  aforementioned  period. 
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EXECUTIVE  SUMMARY 


This  document  will  summarize  the  Operational  Test  and  Evaluation 
(OT&E)  activities  for  the  New  Generation  Runway  Visual  Range 
(RVR)  from  April  1992  to  June  1994.  Testing  outlined  herein 
consisted  of  seven  individual  retests  of  the  RVR  system  conducted 
in  an  operational  setting,  as  well  as  several  specialized  tests 
conducted  in  nonoperational  settings.  The  retests  and 
specialized  tests  followed  initial  OT&E  conducted  in ^February  and 
March  of  1992.  The  RVR  system  was  plagued  by  significant 
discrepancies  in  system  performance  during  initial  OT&E  and 
during  much  of  the  retest  effort.  Despite  numerous  software  and 
hardware  modifications,  resolution  of  the  major  problems 
affecting  the  system  were  not  resolved  to  the  satisfaction  of 
ACW-200  until  early  1994. 

The  New  Generation  RVR  system  is  a  terminal  navigation  aid 
designed  for  use  with  precision  runways  in  support  of  takeoff  and 
landing  operations  during  Category  I,  II,  Illa/b  visibility 
conditions.  The  functions  of  the  RVR  include  determination  of 
the  atmospheric  scattering  coefficient,  the  ambient  luminance, 
and  the  runway  light  intensity.  This  information  is  processed  to 
yield  distances  that  a  pilot  can  expect  to  see  along  the 
departure  or  approach  path  of  a  runway .  The  New  Generation  RVR 
equipment  will  decrease  maintenance  load  and  installation 
difficulties  associated  with  current  RVR  system  designs.  Future 
expansion  capabilities  will  be  easier  and  less  costly. 

The  purpose  of  the  OT&E  testing  was  to  verify  the  RVR  National 
Airspace  System  (NAS)  requirements  as  identified  in  the  NAS 
System  Specification  (NAS-SS-1000) .  The  testing  was  also 
intended  to  verify  the  operational  effectiveness  and  suitability 
of  the  RVR  within  the  NAS  environment. 

While  retests  allowed  evaluations  of  the  RVR  in  an  operational 
environment  (as  required  by  FAA“ORDER~1810.4B) ,  specialized 
testing  permitted  evaluations  of  RVR  sensors  under  real  and 
simulated  weather  conditions  such  as  blowing  precipitation  and 
heavy  fog.  The  specialized  tests  were  instrumental  in  developing 
and  evaluating  software  and  hardware  modifications  aimed  at 
correcting  the  more  difficult  performance  problems. 

Results  of  the  final  retest  conducted  in  June  1994,  as  well  as 
results  from  specialized  tests,  indicate  the  most  significant 
sensor  and  system  problems  have  been  resolved.  This  was 
accomplished  by  permanent  design  changes  as  well  as 
implementation  of  interim  "work-arounds."  As  a  result,  ACW-200B 
recommended  national  deployment  of  the  RVR  system  with  the 
conditions  that  additional  Category  Illb  performance  data  be 
acquired  and  that  problems  currently  having  interim  work-around 
solutions  be  resolved  in  full. 
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1.  INTRODUCTION. 


This  report  will  summarize  the  Operational  Test  and  Evaluation 
(OT&E)  activities  for  the  New  Generation  Runway  Visual  Range  (RVR) 
system  from  February  1992  to  June  1994.  Testing  described  herein 
was  conducted  over  a  26'~month  period  following  the  conclusion  of 
initial  OT&E.  Initial  OT&E  took  place  from  February  to  March  of 
1992,  and  the  results  are  detailed  in  FAA  Technical  Report, 
DOT/FAA/CT-TN92/37. 

During  the  period  covered  by  this  report,  the  RVR  underwent  seven 
separate  retests  as  well  as  specialized  tests  designed  to  evaluate 
sensor  performance.  A  discussion  of  the  results,  problems,  and 
issues  is  provided  for  each  test  in  subseguent  sections  of  this 
report.  This  report  was  developed  in  accordance  with  FAA-STD-024B 
and  FAA-ORDER-1810.4B. 

1.1  PURPOSE  OF  REPORT. 

The  purpose  of  this  report  is  to  provide  results  of  New  Generation 
RVR  OT&E  from  the  period  following  initial  OT&E  testing  to  the 
issuance  of  a  national  deployment  recommendation.  The  report  also 
provides  recommendations  for  future  testing  and  data  collection. 

1.2  SCOPE  OF  REPORT. 

This  report  will  provide  a  summary  of  results  from  retest  and 
specialized  testing  performed  on  the  RVR  following  initial  OT&E. 
Results  of  initial  OT&E  will  not  be  discussed  in  detail;  however, 
significant  findings  from  initial  OT&E  that  relate  to  retest 
objectives  are  discussed.  In  addition  to  individual  test  results, 
this  report  will  discuss  system  changes  which  evolved  as  a  result 
of  problems  identified  during  testing.  Actual  test  procedures  will 
not  be  discussed  in  detail.  Rather,  an  overview  of  each  test 
session  will  be  provided  along  with  significant  findings  and 
conclusions. 

1.3  BACKGROUND. 

The  New  Generation  RVR  has  undergone  extensive  OT&E.  The  primary 
test  site  for  the  RVR  was  the  Kansas  City  International  Airport 
(MCI) .  However,  because  of  problems  identified  during  initial  OT&E 
and  during  accuracy  evaluations  of  the  Visibility  Sensor  (VS) , 
additional  specialized  testing  was  required  at  other  locations  such 
as  the  Cold  Regions  Research  and  Engineering  Laboratory  (CRREL)  in 
Hanover,  NH,  and  Mount  Washington  Observatory,  Mt.  Washington,  NH. 
The  nature  of  the  specialized  testing  is  discussed  in  subsequent 
sections . 
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1.3.1  Initial  OT&E» 


Initial  OT&E  Operational,  Integration,  and  Shakedown  testing  of  the 
RVR  system  took  place  in  February  and  March  of  1992  at  MCI. 
Significant  deficiencies  were  identified  in  the  RyR  system  and  with 
supporting  documentation.  A  summary  of  these  deficiencies  is 
provided  in  paragraph  4. 1.1.1.  As  a  result  of  the  system 
deficiencies,  ACW-200B  recommended  against  deployment  at  the 
conclusion  of  testing. 

1.3.2  Visibility  Sensor  Evaluation. 

As  noted  in  DOT/FAA/CT-TN92/37,  an  evaluation  of  the  RVR  VS  was 
conducted  to  determine  sensor  accuracy .  The  evaluation  was 
conducted  by  personnel  from  the  Volpe  National  Transportation 
System  Center  (VNTSC)  using  data  from  several  test  sites.  A  draft 
report,  DOT-TSC-FAA-92-xx,  was  issued  in  September  1992.  This 
report  was  reviewed  by  ACW“200B  and  AOS“220.  Significant  findings 
of  the  report  included  problems  encountered  with  sensor 
calibration,  accuracy,  and  system  response  to  precipitation.  The 
findings  are  discussed  in  the  following  subparagraphs.  In 
addition,  as  noted  by  AOS-220,  no  significant  data  existed  for 
system  response  in  actual  Category  Illb  visibility  conditions 
(visibility  less  than  700  feet) . 

1.3. 2.1  Calibration. 

Numerous  problems  involving  sensor  calibration  were  noted  during 
the  testing  conducted  by  VNTSC.  Problems  included  disparities 
between  New  Generation  RVR  sensors  and  airport  transmissometers , 
inconsistencies  in  unit  calibrations,  and  apparent  variations  in 
manufacturing  tolerances.  In  addition,  it  was  observed  that  the 
sensor  calibration  for  snow  differed  from  that  of  fog  by 
approximately  30  percent.  This  result  meant  that  system  accuracy 
would  be  affected  if  calibration  was  not  automatically  adjusted  for 
snow  versus  fog  events. 

1 . 3 . 2 . 2  Precipitation  Effects . 

'•Window  Signals"  originating  from  the  RVR's  window  contamination 
detection  circuitry  were  found  to  be  adversely  affected  by 
precipitation  hitting  the  lens  or  "window"  (figure  1)  of  the  VS. 
These  signals  are  actually  voltage  levels  which  are  proportional  to 
the  amount  of  contamination  detected  on  the  sensor  lens.  Window 
contamination  includes  any  substance  such  as  snow,  water,  dirt, 
etc.,  that  might  strike  or  collect  on  the  sensor  window.  Effects 
of  precipitation  striking  the  sensor  window  ranged  from  errors  in 
extinction  coefficients  to  sensor  shutdowns.  Because  of  shutdowns, 
it  was  necessary  to  disable  the  software  modules  involved  in 
contamination  correction.  This  effectively  eliminated  a  major 
functionality  of  the  system. 
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SENSOR  HEADS 


FIGURE  1.  LCXDK-DOWN  VISIBILITY  SENSOR  COMPONENTS 


1.3. 2. 2.1  Snow  Clogging. 


Snow  and  ice  buildup  (figure  2)  on  the  VS  hood  and  window  was 
observed  during  conditions  of  blowing  frozen  precipitation. 

NOTE:  Although  not  specifically  stated  in  the  VNTSC  report,  snow 
and  ice  buildup  not  actually  making  contact  with  the  sensor  lens 
will  not  be  detected  by  window  contamination  circuitry.  The  result 
of  an  undetected  blockage  would  be  higher—than—actual  RVR  readings . 

1.3. 2. 2. 2  Ambient  Light  Sensor. 

Because  the  orientation  of  Ambient  Light  Sensor  (ALS)  optics 
(figure  3)  is  similar  to  that  of  the  VS,  the  ALS  was  also 
susceptible  to  the  problems  noted  in  paragraphs  1. 3.2.2  and 

1.3. 2. 2.1. 


1 . 3 . 2 . 3  Category  Illb  Accuracy . 

Following  the  review  of  the  VNTSC  VS  evaluation  report,  ACW-200  and 
AOS-220  concluded  that  the  lack  of  actual  tests  under  Category  Illb 
conditions  constituted  a  deficiency  in  accuracy  validation  of  the 
system.  ACW-200B  requested  that  additional  testing  be  conducted 
along  with  theoretical  analysis  of  system  response  in  order  to 
quantify  and  validate  Category  Illb  performance. 

1.3.3  Specialized  Testing. 

The  lack  of  naturally  occurring  Category  Illb  conditions,  as  well 
as  infrequent  and  unpredictable  blowing  precipitation  events, 
created  a  need  for  specialized  testing  to  address  problems 
identified  during  VS  accuracy  testing.  Testing  for  snow  clogging 
and  response  was  conducted  with  VS  and  ALS  at  CRREL  in  Hanover,  NH. 
Category  Illb  testing  was  conducted  at  the  Mount  Washington 
Observatory,  Mount  Washington,  NH.  See  paragraph  4.2  for  test 
descriptions  and  results. 

1 . 3 . 3 . 1  Data  Collection  Sites . 

In  order  to  maximize  the  amount  of  collected  data  and  enhance 
understanding  of  RVR  system  performance  under  adverse  conditions, 
automated  data  collection  sites  were  established  at  the  following 
locations: 


a.  Sea-Tac  Airport  (SEA),  WA, 

b.  University  of  Colorado,  Mountain  Research  Station  (MRS) , 
Nederland,  CO, 

c.  St.  John's,  Newfoundland  (YYT) ,  Canada,  and 

d.  Otis  Weather  Test  Facility,  Cape  Cod,  MA. 
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FIGURE  2. 


VISIBILITY  SENSOR  SNOW  CLOGGING 


FIGURE  3 


AMBIENT  LIGHT  SENSOR 


SEA,  YYT,  and  the  Otis  Weather  Test  facility  were  previously 
established  test  sites  used  during  the  original  VS  accuracy  tests 
(paragraph  1.3.2).  MRS  was  established  specifically  for  snow 
calibration  and  clogging  research  following  identification  of 
problems  during  VS  accuracy  tests.  These  sites  were  remotely 
monitored  and  data  was  automatically  retrieved.  The  data  from 
these  sites  represented  VS  and  ALS  performance  during  natural 
weather  phenomena  (i.e.,  blowing  precipitation  and  fog).  This  data 
was  evaluated  as  a  supplement  to  the  data  from  CRREL  and  Mt. 
Washington. 

1.3.4  OT&E  Retest. 

Retest  of  the  RVR  system  began  in  August  of  1992.  Seven  separate 
OT&E  retest  sessions  were  conducted.  Each  retest  involved 
modifications  of  system  software,  and  in  some  cases,  modifications 
to  hardware.  Although  no  specific  procedures  were  conducted  for 
operational  testing,  any  items  affecting  the  operational 
suitability  of  the  system  were  noted  by  the  test  team. 

Specialized  tests  (paragraph  1.3.3)  were  considered  to  be  part  of 
the  overall  OT&E  process;  however,  their  purpose  was  more 
developmental  in  nature  and  typically  focused  on  a  single  aspect  of 
system  performance.  They  are  not  considered  as  retest  in  a  system 
sense.  The  results  of  these  tests  were  crucial  to  correcting  major 
discrepancies  in  the  RVR  system;  therefore,  a  general  description 
of  the  tests  and  results  will  be  presented.  Individual  test 
reports  are  attached  as  noted. 

1.3. 4.1  Test  Schedule /Locations. 

Table  1  provides  a  summary  of  the  designation  and  type  of  testing 
performed,  the  dates  and  location,  and  general  comments  concerning 
the  system  configuration  or  test. 

1.3.5  Design  Modifications. 

There  were  numerous  design  changes  made  to  the  RVR  system 
throughout  OT&E.  These  modifications  were  made  primarily  to  system 
software  in  an  effort  to  correct  ongoing  problems.  After  several 
rounds  of  retest,  as  well  as  additional  specialized  testing,  it 
became  apparent  that  software  modifications  would  not  be  sufficient 
to  eliminate  the  more  serious  problems  affecting  the  RVR  system. 

This  was  particularly  true  for  problems  with  sensor  failure  and 
system  outages  during  blowing  precipitation. 
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TABLE  1.  TESTING  ACTIVITIES 
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As  a  result,  the  feasibility  of  significant  hardware 
modifications  was  studied.  The  results  of  the  studies  indicated 
that  a  combination  of  hardware  and  software  modifications  to  the 
VS  and  ALS  would  be  more  effective  at  eliminating  sensor 
performance  discrepancies. 

The  modified  sensors  and  algorithms  underwent  thorough  testing  at 
CRREL  to  determine  the  extent  of  performance  improvements 
(paragraph  4.2.1) .  Tests  included  performance  under  blowing 
precipitation  and  Category  Illb  accuracy . 

NOTE;  Although  Category  Illb  testing  was  performed  at  CRREL, 
difficulties  were  encountered  in  maintaining  Category  Illb 
conditions  and  discerning  actual  sensor  performance  during 
testing.  As  a  result,  the  Category  Illb  performance  data  was  not 
considered  usable. 

The  original  and  modified  sensor  designs  are  discussed  in  the 
following  subparagraphs. 

1.3. 5.1  Look -Across  Visibility  Sensor. 

The  original  VS  utilized  sensor  heads  that  were  oriented  parallel 
to  the  ground  (figure  4) .  Because  of  this  orientation,  it  is 
referred  to  as  a  "Look -Across”  or  look-out  VS.  OT&E  and 
Specialized  test  results  indicated  that  this  sensor  was 
susceptible  to  calibration  problems,  failures,  and  questionable 
RVR  readings.  The  latter  two  items  being  particularly  prevalent 
during  blowing  rain  or  snow.  Problems  were  attributed  to  the 
relative  ease  with  which  precipitation  could  strike  the  sensor 
window  and  to  the  system's  inability  to  adequately  compensate 
when  this  occurred. 

As  mentioned  in  paragraph  1.3. 2. 2.1,  undetected  contaminants 
blocking  the  infrared  light  path  to  or  from  the  "scatter  volume” 
(figure  1)  can  cause  higher-than-actual  RVR  readings.  This 
phenomenon  was  observed  during  snow  events  which  produced  snow 
"clogs”  (figure  2)  in  the  look-across  sensor. 

1.3. 5. 2  Look-Down  Visibility  Sensor. 

Reevaluation  of  the  original  VS  design  led  to  the  conclusion  that 
it  was  possible  to  orient  the  sensor  heads  in  a  downward- looking 
configuration.  It  was  felt  that  this,  in  conjunction  with  a 
longer  more  enclosed  hood,  would  prevent  precipitation  from 
striking  the  sensor  lenses  under  most  conditions.  Several 
prototypes  of  the  new  design  were  built  to  assess  the 
improvements.  Because  of  the  orientation  of  the  sensor  heads, 
this  version  of  the  VS  is  referred  to  as  the  Look-Down  VS  (figure 

5)  . 
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FIGURE  4.  LOOK-ACROSS  VISIBILITY  SENSOR 


FIGURE  5.  LOOK-DOVJN  VISIBILITY  SENSOR 
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1.3. 5. 2.1  Snow/Rain  Calibration. 


An  added  benefit  of  the  look-down  design  was  the  ability  to 
orient  the  sensor  heads  at  an  angle  that  would  result  in  nearly 
egual  calibration  values  for  snow  and  rain.  This  would  eliminate 
the  need  for  any  automated  process  of  changing  calibration  values 
for  snow  versus  fog.  The  optimum  angle  for  egual  snow  and  fog 
response  was  determined  by  analysis  to  be  42°.  Data  was 
collected  during  specialized  tests  to  verify  correct  calibration 
values. 

1.3. 5. 2. 2  Additional  Changes. 

The  following  modifications  were  also  implemented  in  the  Look- 
Down  VS: 

a.  Improved  lens  filter, 

b.  Resized,  lower  wattage  window  heaters, 

c.  Higher  wattage  hood  heater  with  more  conformal  heat 
distribution,  and 

d.  Addition  of  bird  spike  and  insect  repellents. 

The  objectives  of  these  design  changes  included  the  following: 

a.  Remove  dynamic  components  of  sunlight  that  adversely 
affect  extinction  coefficient  readings; 

b.  Minimize  the  possibility  of  snow  clogging; 

c.  Eliminate  sensor  shutdowns  caused  from  high  window 
contamination ; 

d.  Minimize  the  possibility  and  effects  of  icicle 
formations  on  the  sensor;  and 

e.  Reduce  the  possibility  of  inaccurate  RVR  readings  due 
to  birds  or  insects  entering  the  receiver  or 
transmitter  beam. 

1 . 3 . 5 . 3  Ambient  Light  Sensor . 

Because  the  physical  design  of  the  ALS  is  similar  to  that  of  the 
Look-Across  VS,  it  too  was  susceptible  to  contamination  and 
clogging  problems.  However,  unlike  the  VS,  the  ALS  requires  an 
unobstructed  view  of  the  North  sky  at  a  6°  elevation.  This 
requirement  prevented  any  changes  to  the  sensor  head  orientation. 
Other  hardware  changes  were  incorporated  in  an  effort  to  improve 
the  sensor's  ability  to  respond  to  blowing  precipitation  and  to 
prevent  snow  clogging. 
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Hardware  design  changes  made  to  the  ALS  included  the  following: 

a.  Higher  wattage  heater  with  more  conformal  heat 
distribution; 

b.  Resized  and  lower  wattage  window  heater; 

c.  Elimination  of  "ledge”  under  sensor  head  which  allowed 
snow  to  build  up;  and 

d.  Repositioning  of  grounding  lug  to  the  rear  of  sensor. 
2.  REFERENCE  DOCUMENTS. 


The  following  documents  were  used  in  preparation  of  this  report: 

Preparation  of  Test  and  Evaluation  FAA-STD-024B 

Documentation  August  22,  1994 


FAA  NAS  Test  and  Evaluation  Policy 


FAA-ORDER-1810 . 4B 
October  22,  1994 


Interim  Certification  and  Maintenance  N  6560.16 

of  the  New  Generation  Runway  Visual  Range  November  11,  1994 


Runway  visual  Range  (RVR)  Operational  Test  DOT/FAA/CT-TN92/37 
and  Evaluation  (OT&E)  Integration  and  April  1993 

OT&E  Operational  Test  Report 


RVR  System  On-Site  Reguirements 


Runway  Visual  Range  System  Specification 
Amendment  4 

Functional  and  Performance  Reguirements 
for  the  NAS  Specification 

Remote  Maintenance  Monitoring  System 
Interface  Control  Documents 


TI  6560.17 
September  1994 

FAA-E-2272, 

April  25,  1990 

NAS-SS-1000 
Volumes  III  and  V 

NAS-MD-790,  792,  793 
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3.  SYSTEM  DESCRIPTION. 


3.1  MISSION  REVIEW. 

The  New  Generation  RVR  is  designed  to  replace  transmissometer 
systems  (e.g..  Tasker  400,  500)  currently  in  use  at  U.S. 
airports.  It  will  provide  a  measurement  of  runway  visual  range 
at  specific  points  along  a  precision  runway  in  support  of 
instrument  landings  during  Category  I,  II,  Illa/b  visibility 
conditions  (specification  FAA-E-2772) .  The  functions  of  the  RVR 
include  determination  of  the  following: 

a.  Atmospheric  scattering  coefficients, 

b.  Ambient  light  intensity,  and 

c.  Runway  light  intensity. 

The  above  information  is  processed  to  yield  distances  that  a 
pilot  can  expect  to  see  along  the  departure  or  approach  path  of  a 
runway.  The  New  Generation  RVR  equipment  will  decrease  the 
maintenance  load  and  installation  difficulties  associated  with 
current  RVR  system  designs.  Future  expansion  capabilities  will 
be  easier  and  less  costly. 

3.2  TEST  SYSTEM  CONFIGURATION. 

For  each  retest,  the  RVR  system  at  MCI  consisted  of  the  following 
components  and  interfaces: 

Components 


a.  Visibility  Sensor.  Three^'^  total,  located  at  the 
touchdown,  midpoint,  and  rollout  segments  of  runway; 

b.  Ambient  Light  Sensor.  One  located  on  top  of  the 
airport  control  tower; 

c.  Runway  Light  Intensity  Monitor  (RLIM)  Sensors.  Three 
located  in  the  airport  power  vault; 

d.  Sensor  Interface  Electronics  (SIE)  enclosures.  One 
located  with  each  sensor; 

e.  Data  Processing  Unit  (DPU) .  One  located  in  the 
electronics  equipment  room  of  the  airport  control 
tower ;  and 


Four  visibility  sensors  were  installed  at  MCI  for  Retests  6  and  7. 

2 

Look-across  configuration  sensors  were  used  during  Retests  1  through  5,  look-down  version  was  used 
in  Retests  6  and  7. 
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f.  Controller  Display  (CD).  Four  total,  located  in  the 
control  tower  cab,  the  Terminal  Radar  Control  (TRACON) 
area,  and  the  electronics  equipment  room  of  the  airport 
control  tower;  and 

g.  Firmware  for  the  DPU,  VS,  ALS,  CD,  and  RLIM. 


Interfaces 


a.  Maintenance  Data  Terminal  (MDT) .  MDT  interfaces  were 
contained  in  SIE  enclosures  and  the  DPU; 

b.  Maintenance  Processor  Subsystem  (MPS) .  The  MPS 
interface  was  contained  within  the  DPU;  and 

c.  External  User  (EU) .  The  EU  port  was  contained  within 
the  DPU. 

Specialized  test  results  from  CRREL  and  Mt.  Washinqton  are 
discussed  in  paragraphs  4.2.1  and  4.2.2.  Refer  to  these 
paragraphs  for  a  description  of  the  system  configuration  during 

testing. 

3.3  ADDITIONAL  INTERFACES. 

Testing  of  the  Tower  Control  Computer  Complex  (TCCC) ^and  the 
Automated  Surface  Observing  System  (ASOS)  National  Airspace 
System  (NAS)  subsystem  interfaces  was  deferred.  The  interfaces 
were  not  available  at  the  time  of  testing. 


4.  TEST  AND  EVALUATION  DESCRIPTION. 


This  section  describes  retest  and  specialized  testing  conducted 
on  the  RVR  following  completion  of  initial  OT&E.  Retests  are 
presented  in  chronological  order  along  with  results.  Specialized 
tests  are  presented  after  the  retests.  See  referenced  reports 
for  detailed  descriptions  of  test  procedures  and  results. 

In  addition  to  the  testing  discussed  herein,  system  performance 
was  remotely  monitored  by  AOS-220  during  the  periods  between 
retests.  System  anomalies  discovered  during  these  periods  will 
be  included  in  retest  discussions. 
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4.1  RETESTS. 


Retests  were  conducted  to  verify  resolution  of  problems  found 
during  initial  OT&E  or  subsequent  tests.  The  test  philosophy  was 
to  conduct  individual  procedures  to  verify  correction  or 
continued  existence  of  each  discrepancy .  Upon  completion  of 
individual  tests,  overall  system  functionality  was  checked  to 
ensure  that  modifications  had  not  affected  other  areas  of  system 
operation . 

Tests  were  conducted  using  a  subset  of  the  original  integration 
and  shakedown  procedures.  Additional  procedures  were  developed 
during  testing  when  warranted. 

Information  and  pass /fail  requirements  for  additional  test 
procedures  were  derived  from  the  following  documents: 

a.  '  TI  6560.17, 

b.  FAA-E-2272, 

c.  NAS-SS-1000,  and 

d.  NAS-MD-790/NAS-MD-792/NAS-MD-793 . 

4.1.1  Retest  1. 

The  following  subparagraphs  describe  the  conduct  and  results  of 
the  first  retest  performed  on  the  RVR  system  following  initial 
OT&E.  Testing  was  intended  to  evaluate  changes  made  in  response 
to  the  initial  OT&E  report.  Reference  appendix  A  for  individual 
test  reports. 

4. 1.1.1  Background. 

RVR  modifications  consisted  primarily  of  system  software  changes. 
System  software  was  delivered  for  Retest  1  as  an  engineering 
prototype  and  was  not  officially  released. 

Initial  OT&E  results  indicated  significant  problems  in  the  areas 
listed  below.  Additional  details  pertaining  to  the  problem  areas 
are  provided  in  the  following  subparagraphs: 

a.  Sensor  performance, 

b.  System  documentation, 

c .  Maintenance , 

d.  Personnel  safety, 

e.  Remote  Maintenance  Monitoring  (RMM) ,  and 

f.  Controller  Display. 
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4. 1.1. 1.1  Sensor  Performance. 


Discrepancies  which  would  affect  system  accuracy  were  observed 
during  initial  OT&E  and  during  VS  evaluations  (paragraph  1.3.2;. 
The  more  significant  problems  appeared  to  be  „ 

precipitation,  and  affected  the  operation  of  both  the  VS  and  ALS. 
The  discrepancies  included  fluctuating  extinction  coefficient 
readings  and  occasional  sensor  shutdowns. 


Inconsistencies  with  sensor  calibration  were  also  noted  during 
the  course  of  testing.  Calibration  problems  included  an  unequal 
response  of  the  sensor  during  snow  and  fog  conditions.  This 
resulted  in  inaccurate  RVR  readings  during  snow  events. 


Problems  with  the  RLIM  were  also  observed  during  initial  OT&E  at 
MCI.  These  problems  were  related  to  an  apparent  violation  of 
fail-safe  requirements  and  incomplete  fault  detection  for  the 
RLIM  sensor. 


In  addition,  after  reviewing  results  of  VNTSC  sensor  evaluations, 
AOS-220  noted  that  data  collected  did  not  include  occurrences  of 
Category  Illb  conditions;  therefore,  no  data  existed  which 
validated  sensor  performance  in  this  visibility  range. 


4. 1.1. 1.2  System  Documentation. 

In  several  areas,  the  On— Site  Manual  (TI  6560.17)  was 
insufficient  to  determine  if  RVR  operation  was  correct.  These 
areas  included  maintenance  procedures  along  with  sensor  operation 
and  performance.  Appropriate  safety  warnings  were  missing  from 
the  documentation,  as  well  as  procedures  for  handling  failures  in 
RVR  components.  Additionally,  the  Off-Site  Manual  (TI  6560.18) 
was  unavailable  for  evaluation. 


4. 1.1. 1.3  Maintenance. 

Significant  maintenance  problems  found  included  the  following: 

a.  Insufficient  monitoring  of  SIE  batteries; 

b.  Loss  of  archived  data; 

c.  Violation  of  fail-safe  requirements  and  inability  to 
detect  faults  in  RLIM  sensors; 

d.  Insufficient  performance  capabilities  of  lightning 
protection  circuitry  and  the  inability  to  detect 
failures  in  these  components; 

e.  Difficulty  in  accessing  DPU  power  supply  test  points; 


15 


f.  Display  of  invalid  products  at  the  CD  without  proper 
notification;  and 

g.  Excessive  corrosion  of  SIE  enclosures. 

4. 1.1. 1.4  Remote  Maintenance  Monitoring. 

Problems  in  this  area  included  system  control  conflicts  between 
the  MPS  and  MDT,  and  loss  of  communication  and  data  when  the 
system  was  on  battery  power. 

4. 1.1. 1.5  Personnel  Safety. 

Safety  issues  included  electrical  shock  hazards  in  SIE  enclosures 
and  hazards  to  technicians  when  raising  and  lowering  VS  for 
maintenance. 

4. 1.1. 1.6  Controller  Display. 

Problems  noted  with  the  CD  included  the  followingj 

a.  Misrepresentation  of  RVR  products; 

b.  Ability  to  inadvertently  change  display  configurations; 

c.  Loss  of  data  during  power  fluctuations; 

d.  Failures  in  self -test  operations; 

e.  Day /night  readability;  and 

f.  Lack  of  alarm  capability  when  system  failures  were 
present . 

4 . 1 . 1 . 2  Test  Date  and  Location . 

Testing  was  performed  from  August  17  through  August  21 
(Integration) ,  and  from  September  1  through  September  4 
(Shakedown),  1992.  Tests  were  conducted  at  MCI,  Kansas  City, 
Missouri. 
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4. 1.1. 3  Participants. 


Personnel  from  the  following  organizations  conducted  and 
supported  OT&E  retest: 

Organization  Role 


ACW-200B 

ACN-IOOD 

ANN-140 

MCI  Airway  Facilities 
Sector  Field  Office  (AFSFO) 
Kansas  City  Air  Route  Traffic 
Control  Center 


AOS-220 


Test  Director 
Test  Engineering 
Observer 

Technicians 

System  Administrators 
Shakedown  Test  Engineering 


4. 1.1. 4  Specialized  Test  Equipment. 

Test  eguipment  included  protocol  analyzers  and  remote  maintenance 
simulation  software.  No  additional  specialized  test  equipment 
was  required. 

4. 1.1. 5  Test  Objectives. 

The  primary  objectives  of  testing  were  to  verify  corrections  to 
problems  previously  identified  and  to  ensure  the  operational 
suitability  of  the  RVR  system.  This  round  of  testing  was 
intended  to  verify  improvements  and  performance  in  the  areas 
noted  in  paragraph  4 . 1 . 1 . 1 . 

4. 1.1. 6  Test  Descriptions. 

Test  procedures  were  essentially  a  subset  of  those  performed 
during  initial  OT&E.  When  necessary,  procedures  were  changed  to 
isolate  system  responses  and  problems. 

When  possible,  conditions  under  which  problems  were  first 
identified  were  reproduced.  If  successive  attempts  to  duplicate 
a  problem  failed  and  no  degraded  performance  was  observed,  the 
discrepancy  was  identified  as  resolved. 

4. 1.1. 7  Data  Collection  and  Analysis. 

Test  conduct  forms  were  maintained  during  testing  and  discrepancy 
reports  were  created  as  necessary.  Data  communication  log  files 
were  also  established  for  analysis  purposes. 
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4. 1.1.8  Test  Results. 


Test  reports  for  Retest  1  can  be  found  in  appendix  A. 


With  the  exception  of  some  improvements  in  RMM  and  fault 
detection,  the  major  problems  noted  during  initial  OT&E  remained 
relatively  unchanged. 


The  improvements  in  RMM  included  elimination  of  local/remote 
control  conflicts,  as  well  as  correction  of  other  minor 
discrepancies.  Fault  detection  improvements  included  the 
addition  of  the  following: 


a.  RLIM  SIE  link  error  values, 

b.  Hard  Alarms  when  AC  power  is  lost, 

c.  Hard  Alarm  threshold  values  for  DPU  power  supplies,  and 

d.  Site  Data  reports  after  issuance  of  fault  diagnostic 
commands . 

The  more  significant  systematic  deficiencies  remained 
uncorrected.  Test  results  indicated  existing  and/or  new  problems 
in  the  following  areas: 


a.  Sensor  Performance, 

b.  System  Documentation, 

c.  Maintenance, 

d.  Personnel  Safety, 

e.  Remote  Maintenance  Monitoring,  and 

f.  Controller  Display. 

Additional  details  are  provided  in  the  following  subparagraphs. 


4. 1.1. 8.1  Sensor  Performance. 

Discrepancies  in  VS  and  overall  RVR  accuracy  were  observed. 
Performance  was  particularly  suspect  during  the  following 
conditions: 

a.  VS  lens  contamination;  and 

b.  Transitions  from  day /night  operation. 

Discrepancies  were  also  noted  in  the  operation  of  rain  and  snow 
filters.  These  filters  are  incorporated  through  software  and 
were  intended  to  allow  the  system  to  compensate  for  transient 
window  signals  (paragraph  1.3. 2. 2)  caused  by  precipitation 
hitting  the  sensor  lenses. 
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4. 1.1. 8.2  System  Documentation. 

System  documentation  had  not  been  updated.  Discrepancies  were 
essentially  the  same  as  noted  in  paragraph  4. 1.1. 1.2.  In 
addition,  updates  were  missing  from  the  CD  manual. 

4. 1.1. 8. 3  Maintenance. 

Although  improvements  were  made  in  fault  diagnostic  software, 
fault  detection  and  fail-safe  protection  for  the  RLIM  was  still 
deemed  inadequate. 

Other  problems  noted  included  the  following; 

a .  Use  of  the  MDT  to  enter  or  read  system  parameters  was 
complicated  by  the  need  to  use  a  calculator  to  convert 
the  information  to  a  usable  format  using  scaling 
factors ;  and 

b.  One-hour  archiving  function  entered  an  endless  loop. 

Previously  reported  problems  remained  as  stated  in  paragraph 

4. 1.1. 1.3. 

4. 1.1. 8. 4  Remote  Maintenance  Monitoring. 

As  discussed  in  paragraph  4. 1.1. 8,  some  improvements  were  noted 
in  RMM  performance.  However,  new  problems  were  observed. 

Problems  occurred  in  the  following  areas: 

a.  Communication  failures  between  the  Maintenance 
Processor  Subsystem  (MPS)  and  the  RVR  Remote  Monitoring 
Subsystem  (RMS) ; 

b.  Command  execution  and  format  failures  during  MPS  to  RVR 
RMS  communication; 

c.  Failures  in  alarm  generation;  and 

d.  Failure  to  clear  alarm  conditions  as  required  by 
specification . 

4. 1.1. 8. 5  Personnel  Safety. 

Problems  remained  as  stated  in  paragraph  4. 1.1. 1.5. 

4. 1.1. 8. 6  Controller  Display. 

Previous  problems  noted  with  the  CD  remained.  An  additional 
discrepancy  discovered  was  the  absence  of  audible  alarms  at  the 
CD  when  RVR  equipment  failures  existed. 
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4. 1.1. 9  Recominendations. 


Based  on  the  continued  existence  and  seriousness  of  problems 
observed  during  initial  OT&E,  as  well  as  newly  discovered 
anomalies,  ACW-200B  recommended  that  the  RVR  system  not  be 
approved  for  national  deployment. 

4.1.2  Retest  2. 

The  following  subparagraphs  describe  the  conduct  and  results  of 
the  second  retest  performed  on  the  RVR  system.  Testing  was 
intended  to  evaluate  RVR  modifications  made  in  response  to 
results  from  initial  OT&E  and  Retest  1.  Reference  appendix  B  for 
individual  test  reports. 


4. 1.2.1  Background. 

Following  Retest  1,  modifications  were  made  to  RVR  system 
software  to  correct  existing  deficiencies  (paragraph  4. 1.1. 8}. 
Significant  in  the  changes  were  modifications  to  the  rain  an 
snow  filters.  The  modified  software  was  delivered  for  Retest 
as  an  engineering  prototype  and  was  not  officially  released. 


4. 1.2. 2  Test  Date  and  Location. 

Testing  was  conducted  November  9  through  November  17 
(Integration) ,  and  December  1  through  December  11  (Shakedown) , 
1992,  at  MCI,  Kansas  City,  Missouri. 


4 . 1 . 2 . 3  Participants . 

Personnel  from  the  following  organizations  conducted  and 
supported  OT&E  retest: 

Organization  Role 

Test  Director 
Test  Engineering 
Observer 

Shakedown  Test  Engineering 
Technicians 

Route  Traffic 

System  Administrators 

4 . 1 . 2 . 4  Specialized  Test  Eguioment . 

Test  equipment  included  protocol  analyzers  and  remote  maintenance 
simulation  software. 


ACW-200B 

ACN-IOOD 

ANN-140 

AOS-220 

MCI  AFSFO 

Kansas  City  Air 

Control  Center 
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4. 1.2. 5  Test  Objectives. 


Test  objectives  were  the  same  as  those  stated  in  paragraph 
4. 1.1.5.  This  round  of  testing  was  intended  to  verify 
improvements  and  performance  in  the  discrepancy  areas  noted  in 
Retest  1  results.  More  specifically,  the  objectives  were  to 
determine  the  status  of  issues  in  the  following  areas: 


a. 

Sensor  Performance, 

b. 

System  Documentation, 

c. 

Maintenance, 

d. 

Personnel  Safety, 

e. 

Remote  Maintenance  Monitoring,  and 

f . 

Controller  Display. 

4. 1.2. 6 

Test  Descriotions. 

Reference  paragraph  4. 1.1. 6. 

4. 1.2. 7 

Data  Collection  and  Analvsis. 

Test  conduct  forms  were  maintained  during  testing  and  discrepancy 
reports  were  created  as  issues  occurred.  Data  communication  log 
files  were  also  established  for  analysis  purposes. 

4. 1.2. 8  Test  Results. 

Test  results  indicated  some  improvements  in  sensor  performance 
and  maintainability. 

Sensor  improvements  were  noted  in  the  improved  ability  of  the  VS 
to  compensate  for  lens  contaminants;  however,  the  RVR  system 
continued  to  malfunction  during  real  and  simulated  precipitation 
events . 

The  addition  of  RLIM  configuration  parameters  and  the  correction 
of  errors  in  product  archiving  aided  in  improving  system 
maintainability.  Other  maintenance  improvements  included  the 
retention  of  sensor  calibration  data  during  system  restarts  and 
shutdowns . 

Minor  problems  were  corrected  in  the  RMM  area.  Progress  included 
the  closing  of  trouble  reports  in  the  following  areas: 

a.  MPS  to  RMS  communications, 

b.  Execution  of  fault  diagnostic  commands, 

c.  Alarm  generation  and  clearing,  and 

d.  RMS  Message  formatting. 
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Test  results  also  indicated  existing  and/or  new  problems  in  the 
following  areas: 

a.  Sensor  Performance, 

b.  System  Documentation, 

c.  Maintenance, 

d.  Remote  Maintenance  Monitoring, 

e.  Personnel  Safety,  and 

f.  Controller  Display. 

Additional  details  are  provided  in  the  following  subparagraphs. 

4. 1.2. 8.1  Sensor  Performance. 

Despite  modifications  to  rain  and  snow  filters,  critical  failures 
were  observed  in  th®  VS  and  ALS.  Failures  included  loss  of 
calibration  during  normal  operation,  and  sensor  shutdowns  for 
extended  periods.  Of  particular  note  was  a  4“hour  outage^ 
recorded  during  a  snow  event  and  two  separate  outages  during 
blowing  rain  conditions.  During  the  snow  event,  the  existing 
transmissometer  system  remained  operational  and  recorded  minimum 
visibility  between  2,000  and  3,000  feet.  The  shutdowns  were 
attributed  to  high  window  signals  caused  by  precipitation 
striking  the  VS  and  ALS  lenses. 

Additional  operational  failures  were  noted  in  VS  hood  heaters. 

The  hood  heaters  failed  to  activate  at  required  temperatures. 

Other  previously  identified  problems  and  issues  (paragraph 
4. 1.1. 8.1)  remained. 

4 . 1 . 2 . 8 . 2  System  Documentation . 

System  documentation  problems  remained  as  stated  in  paragraph 
4. 1.1. 8. 2.  Additional  documentation  deficiencies  were  noted  and 
included  missing  information  for  SIE  MDT  screens. 

4. 1.2. 8. 3  Maintenance. 

As  mentioned  in  4. 1.2. 8,  corrections  were  made  to  the  data 
archiving  function;  however,  additional  maintenance  related 
problems  were  observed.  These  included: 

a.  Communication  failure  between  MDT  and  DPU/SIE; 

b.  Incorrect  operation  of  fault  diagnostic  tests; 

c.  Improper  MDT/SIE  configuration  screen  identification; 

d.  Failure  to  regain  sensor  calibration  data  after  power 
outages ;  and 
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e.  Improper  operation  of  function  keys. 

With  the  exception  of  data  archiving,  the  problems  as  identified 
in  paragraph  4. 1.1. 8. 3  remained  unresolved. 

4. 1.2. 8. 4  Remote  Maintenance  Monitoring. 

Despite  progress  noted,  additional  problems  were  discovered  with 
the  remote  maintenance  subsystem  interface. 

The  additional  problems  found  included  the  following: 

a.  The  RVR  RMS  intermittently  sends  data  frames  to  the  MRS 
instead  of  sending  one  continuous  data  stream;  and 

b.  The  RMS  failed  to  provide  the  proper  response  while  in 
"Disconnect"  Mode. 

4 . 1 . 2 . 8 . 5  Personnel  Safety . 

All  personnel  safety  problems  remained  as  stated  in  paragraph 

4. 1.1. 1.5. 


4. 1.2. 8. 6  Controller  Display. 

CD  remained  as  stated  in  paragraph  4. 1.1. 8. 6. 

4. 1.2. 9  Recommendations. 

The  corrections  and  modifications  made  to  the  RVR  software 
produced  only  marginal  improvements  in  the  areas  containing 
critical  performance  problems.  As  a  result,  ACW-200  again 
recommended  against  a  national  deployment  decision. 

4.1.3  Retest  3 . 

Test  description  and  results  of  Retest  3  are  discussed  in  the 
following  subparagraphs.  As  in  previous  tests,  the  purpose  of 
Retest  3  was  to  verify  the  most  recent  modifications  made  to  the 
RVR  system.  Reference  appendix  C  for  individual  test  reports. 
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4. 1.3.1  Background. 

In  the  period  between  Retest  2  and  Retest  3,  an  officially 
released  version  of  software  was  installed  in  the  RVR  sptem  at 
MCI  airport.  This  software  included  modifications  which  allowed 
for  an  increase  in  the  number  of  VSs.  The  expansion  was 
incorporated  to  support  RVR  system  installations  at  larger 
airports  such  as  the  new  Denver  International  Airport  (DVX) .  Tne 
expansion  increased  the  RVR's  VS  capacity  from  12  to  18. 

Although  RVR  sensor  capacity  was  increased,  no  additional  vs  were 

installed  at  MCI. 

Failure  to  obtain  a  deployment  recommendation  for  the  RVR  after 
the  second  retest  elevated  program  office  concerns  related  to 
having  a  system  available  for  the  opening  of  the  new  DVX.  As  a 
result,  efforts  became  more  focused  on  fixing  or  finding  work¬ 
arounds  for  the  major  sensor  problems.  ACW-200  accepted  this 
approach  with  the  stipulation  that  Denver  be  the  only  site  to 
receive  the  system,  and  that  the  Denver  RVR  would  be  upgraded  as 
soon  as  a  national  deployment  configuration  was  achieved. 
Modifications  to  the  RVR  software  following  Retest  2  reflected 
the  new  focus,  and  the  version  was  dubbed  the  "Denver 
Configuration. " 


4. 1.3. 2  Test  Date  and  Location. 

Testing  was  conducted  during  the  following  periods:  from  June  14 
through  June  18  (Shakedown) ,  and  June  18  through  June  25 
(Integration),  1993,  at  MCI,  Kansas  City,  Missouri. 


4. 1.3. 3  Participants. 


Personnel  from  the  following  organizations  conducted  and 
supported  OT&E  retest: 


Organ i z a t i on 

ACW-200B 

ACN-IOOD 

AOS-220 

ANN-140 

MCI  Airway  Facilities 
Sector  Field  Office  (AFSFO) 
Kansas  City  Air  Route  Traffic 
Control  Center 


Role 

Test  Director 
Test  Engineering 
Shakedown  Test  Engineering 
Observer 

Technicians 

System  Administrators 


4 . 1 . 3 . 4  Specialized  Test  Equipment . 

Test  equipment  included  protocol  analyzers  and  remote  maintenance 
simulation  software. 
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4. 1.3. 5  Test  Objectives. 


Test  objectives  were  essentially  the  same  as  those  stated  in 
paragraph  4. 1.1.5.  This  round  of  testing  was  intended  to  verify 
improvements  and  performance  in  the  areas  noted  in  paragraph 
4. 1.2. 8,  with  emphasis  on  items  preventing  deployment  to  Denver. 
In  addition,  testing  was  intended  to  define  the  extent  of 
performance  degradation  to  be  expected  as  a  result  of  deploying 
the  system  with  known  discrepancies.  Results  would  be  provided 
to  airport  personnel  prior  to  system  commissioning. 

4. 1.3. 6  Test  Descriptions. 

Reference  paragraph  4. 1.1. 6. 

4. 1.3. 7  Data  Collection  and  Analysis. 

Test  conduct  forms  were  maintained  during  testing  and  discrepancy 
reports  were  created  as  issues  occurred.  Data  communication  log 
files  were  also  established  for  analysis  purposes. 

4. 1.3.8  Test  Results. 


Sensor  performance  was  again  found  to  be  unacceptable  during 
testing.  The  RVR  experienced  rain  related  shutdowns  as  described 
in  following  subparagraphs. 

Improvements  were  noted  in  the  maintenance  and  RMM  areas.  MDT 
screens  relating  to  RVR  product/edit  override  capabilities  were 
corrected.  Improvements  were  made  in  communications  between  the 
MDT  and  the  RVR  DPU  when  performing  configuration  changes. 
Progress  in  the  RMM  area  included  the  completion  of  command  error 
messages  and  the  resolution  of  some  MPS  software  issues. 

Test  results  again  indicated  continuing  problems  in  the  following 
areas : 

a.  Sensor  performance, 

b.  System  documentation, 

c.  Maintenance, 

d.  Personnel  safety, 

e.  Remote  Maintenance  Monitoring,  and 

f.  Controller  Display. 

Additional  details  are  provided  in  the  following  subparagraphs. 
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4.1.3.8>1  Sensor  Performance. 


As  in  previous  tests,  system  shutdowns  due  to  sensor  failure  were 
observed.  In  particular,  a  system  shutdown  lasting  for  several 
hours  occurred  during  a  rain  event  at  the  airport. 
transmissometer  based  RVR  system  remained  operational  throughout 
the  event,  and  airport  operations  were  unchanged.  The  shutdown 
was  later  attributed  to  a  failure  in  the  ALS  caused  when  rain 
struck  the  lens  of  the  sensor . 


4.1. 3.8.2  System  Documentation. 

The  Off-Site  Technical  Instruction  book  was  not  available  for 
review.  Corrections  to  the  On-Site  Technical  Instruction  manual 
had  not  been  accomplished. 


4. 1.3. 8. 3  Maintenance. 

Maintenance  problems  remained  as  stated  in  paragraph  4. 1.2. 8. 3. 
Additionally,  incorrect  values  were  noted  for  two  data  points 
within  the  VS  SIE  parameter  screens.  The  incorrect  values 
occurred  while  the  parameters  were  in  an  alarm  state. 

4. 1.3. 8. 4  Remote  Maintenance  Monitoring. 

Despite  improvements  in  RMM  as  stated  in  paragraph  4. 1.3. 8, 
additional  problems  were  observed  during  testing.  Significant 
problems  included  the  following; 

a.  RMS  tendency  to  reset  during  the  execution  of  remote 
control  coitimands;  and 

b.  Incorrectly  prioritizing  RMS  messages  to  MRS. 

4. 1.3. 8. 5  Personnel  Safety. 

Previously  observed  personnel  safety  problems  remained  as  stated 
in  paragraph  4 . 1 . 1 . 1 . 5 . 

4. 1.3. 8. 6  Controller  Display. 

CD  discrepancies  remained  as  stated  in  paragraph  4. 1.1. 8. 6.  An 
additional  discrepancy  was  noted  involving  alarms  sounding  when 
runway  light  settings  were  less  than  step  3. 
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4 . 1 . 3 . 9  Recommendations . 


In  response  to  the  test  results,  ACW-200B  recommended  against 
limited  or  national  deployment  of  the  RVR  system.  AOS—220  and 
ACW-200B  also  recommended  that,  following  modifications,  the  RVR 
be  retested  specifically  for  performance  under  blowing 
precipitation  conditions.  It  was  also  recommended  that  accuracy 
testing  be  performed  to  validate  RVR  performance  under  Category 
Illa/b  conditions. 

4.1.4  Retest  4 . 

The  following  subparagraphs  detail  the  test  and  results  of  the 
fourth  retest  performed  on  the  RVR.  Testing  was  intended  to 
verify  system  modifications  designed  to  support  RVR  operations  at 
DVX.  Refer  to  appendix  D  for  individual  reports. 

4 . 1 . 4 . 1  Background . 

Modifications  made  in  preparation  for  this  test  were  aimed 
primarily  at  providing  a  system  which  was  "operationally 
acceptable"  for  use  at  DVX.  At  the  time  of  the  test,  DVX  was 
scheduled  for  opening  in  December  1993. 

Operationally  acceptable  was  defined  by  the  following  conditions: 

a.  Work-arounds  would  be  implemented  for  significant 
problems  that  could  not  be  corrected  prior  to 
deployment  at  DVX; 

b.  Description  of  problems  and  their  operational  impact 
delivered  to  appropriate  users  and  technicians; 

c.  Completion  of  a  full  retest  of  the  RVR;  and 

d.  No  additional  significant  problem  areas  discovered 
during  a  full  retest. 

Consideration  for  deploying  the  RVR  system  at  DVX  in  a  somewhat 
degraded  capacity  was  made  necessary  by  the  lack  of  an  alternate 
means  of  providing  an  RVR  capability  at  the  new  airport.  The  RVR 
system  was  not  considered  by  ACW-200  to  be  ready  for  national 
deployment.  ACN-IOOD  and  AOS-220  were  in  agreement  with  this 
assessment . 

At  the  time  of  the  test,  additional  performance  evaluations  were 
underway  at  Teledyne  Controls ,  Inc . ,  and  CRREL .  The  intent  of 
these  evaluations  was  to  facilitate  rapid  prototype  and 
assessment  of  system  modifications  in  an  effort  to  provide  an 
acceptable  system  for  DVX  as  well  as  national  deployment. 
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Test  and  modification  efforts  which  followed  Retest  3  were 
directed  at  problem  areas  not  related  to  the  RMS  interface.  As  a 
result,  ACW-200B  made  the  decision  to  conduct  only  OT&E  Shakedown 
tests  for  Retest  4.  It  was  expected  that  a  limited  deployment 
recommendation  for  DVX  would  follow  a  full  retest  to  be  conducted 
at  a  later  date. 

In  response  to  RVR  shutdowns  that  occurred  during  the  previous 
retest,  additional  modifications  were  made  to  the  software 
involved  in  lens  contamination  response  of  the  ALS  and  VS .  The 
changes  were  designed  to  allow  the  sensors  to  be  less  sensitive 
to  the  effects  of  precipitation  striking  the  sensor  window. 

In  particular,  the  modifications  consisted  of  specific  additions 
to  the  window  contamination  algorithms.  The  additions  are 
briefly  described  below: 

a.  "Signal  variance"  precipitation  detection, 

b.  Precipitation  detection  periods,  and 

c.  Alarm  delay  periods. 

4. 1.4. 1.1  signal  Variance  Precipitation  Detection. 

The  signal  variance  precipitation  detection  is  a  measurement  of 
the  change  in  window  signals  over  a  specified  "precipitation 
detection  period."  The  signal  variance  is  used  to  determine  the 
"mode"  of  the  sensor.  If  the  window  signals  of  the  VS  or  ALS  are 
changing  rapidly,  it  is  assumed  to  be  a  transient  type  of 
contamination,  e.g.,  rain.  The  "precipitation  mode"  is  then 
enabled.  Conversely,  if  the  change  in  the  window  signals  is 
gradual,  as  would  be  expected  with  the  slow  buildup  of  dust  or 
salt,  the  "dirt  mode"  is  enabled.  Normal  operation  of  the  sensor 
would  be  the  dirt  mode.  Enabling  the  precipitation  mode 
introduces  alarm  delays  and  filtering  to  prevent  the  sensor  from 
overreacting  to  the  transient  signals  caused  by  precipitation 
striking  sensor  lenses. 

4. 1.4. 1.2  Precipitation  Detection  Period. 

The  contamination  algorithm  measures  window  signal  variance  over 
a  span  of  time  which  is  designated  the  Precipitation  Detection 
Period.  This  detection  period  is  a  parameter  that  is  configured 
via  the  DPU  MDT.  The  default  value  is  3  minutes. 
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4. 1.4. 1.3  Alarm  Delay  Period. 


The  alarm  delay  period  is  also  part  of  the  window  contamination 
algorithm.  The  delay  period  allows  window  signal  levels  to 
temporarily  exceed  thresholds  without  triggering  alarms.  Signals 
remaining  above  thresholds  beyond  the  delay  period  will  trigger 
contamination  alarms.  Extremely  large  window  signals  will 
trigger  alarms  immediately.  Like  the  precipitation  detection 
period,  the  alarm  delay  period  is  configurable  via  the  DPU  MDT. 

4. 1.4. 2  Test  Date  and  Location. 

Testing  was  conducted  from  August  16  through  August  20,  1993,  at 
MCI,  Kansas  City,  Missouri.  OT&E  Shakedown  tests  were  conducted 
during  this  period. 


4. 1.4. 3  Participants. 

Personnel  from  the  following 
supported  OT&E  retest: 

Organization 

ACW-200B 
AOS-220 
MCI  AFSFO 


organizations  conducted  and 


Role 

Test  Director 

Shakedown  Test  Engineering 
Technicians 


4. 1.4. 4  Specialized  Test  Equipment. 

No  specialized  test  equipment  was  required. 

4. 1.4. 5  Test  Obiectives. 

The  primary  objective  of  this  test  was  to  assess  the 
effectiveness  of  modifications  made  to  the  ALS  and  VS.  Testing 
concentrated  on  evaluation  of  sensor  performance  in  the  presence 
of  precipitation  and  window  contamination.  Modifications  to  the 
On-Site  Technical  Instruction  manual  were  also  reviewed  during 
the  test  period.  Other  previously  identified  problems  and  issues 
were  not  evaluated  during  this  period. 

4. 1.4. 6  Test  Descriptions. 

Testing  included  performing  calibration  procedures  and  observing 
sensor  performance  under  simulated  precipitation  conditions. 

4. 1.4. 7  Data  Collection  and  Analysis. 

Test  conduct  forms  were  maintained  during  testing  and  discrepancy 
reports  were  created  as  issues  occurred. 
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4. 1.4. 8  Test  Results. 


Some  difficulties  were  encountered  with  sensor  calibration  during 
testing;  however,  test  results  indicated  an  apparent  performance 
improvement  during  simulated  precipitation  conditions.  In 
particular,  the  operation  of  the  precipitation  detection  and 
alarm  delay  periods,  and  the  signal  variance  precipitation 
detection  appeared  to  reduce  sensor  shutdowns  during  test 
conditions. 

Although  test  results  indicated  significant  improvement,  results 
were  inconclusive,  because  sensor  performance  under  actual 
precipitation  conditions  was  not  observed.  More  extensive 
testing  was  recommended  by  ACW— 200  and  AOS—220  to  determine  if 
sensor  performance  would  hold  up  under  all  weather  conditions 
experienced  at  U.S.  airports. 

Test  results  indicated  continuing  problems  in  the  following 
areas : 


a.  Sensor  performance,  and 

b.  System  documentation. 

The  following  areas  were  not  evaluated  during  test  and  were 
assumed  to  be  unchanged: 

a.  Maintenance, 

b.  Personnel  safety, 

c.  Remote  Maintenance  Monitoring,  and 

d.  Controller  Display. 

Additional  details  are  provided  in  the  following  subparagraphs. 

4 . 1 . 4 . 8 . 1  Sensor  Performance . 

Significant  problems  encountered  during  testing  included  an 
unacceptable  number  of  calibration  attempts  before  calibration 
was  successful.  Discrepancies  were  also  noted  with  the 
calibration  apparatus  and  in  the  documentation  supporting  the 
calibration  procedure. 

Inconsistencies  were  also  noted  for  the  VS  and  ALS  window 
contamination  gain  parameters.  The  actual  values  for  these 
parameters  did  not  match  the  default  settings  contained  in  the 
software. 

4. 1.4. 8. 2  System  Documentation. 

Significant  updates  were  made  to  the  On-Site  Technical 
Instruction  manual.  However,  it  was  noted  that  additional 
modifications  to  this  documentation  were  required. 


30 


4 ♦1.4. 8. 3  Maintenance. 


Maintenance  problems  remained  as  stated  in  paragraph  4. 1.3. 8. 3. 

4. 1.4. 8. 4  Remote  Maintenance  Monitoring. 

RMM  problems  were  not  evaluated  during  testing. 

4. 1.4. 8. 5  Personnel  Safety. 

Personnel  safety  problems  remained  as  stated  in  paragraph 

4. 1.1. 8. 5. 

4. 1.4. 8. 6  Controller  Display. 

The  CD  was  not  evaluated  during  testing. 

4 . 1 . 4 . 9  Recommendations ♦ 

The  following  recommendations  were  made  by  ACW-200  and  AOS-220 
following  the  completion  of  testing; 

a.  Conduct  more  extensive  tests  during  simulated  and 
actual  blowing  precipitation  conditions  to  fully 
evaluate  sensor  performance  with  the  newly  modified 
window  contamination  algorithms; 

b.  Investigate  and  correct  difficulties  encountered  during 
the  calibration  of  the  VS;  and 

c.  Correct  and  update  the  Technical  Instruction  manual. 

As  mentioned  previously,  this  was  not  a  complete  retest.  A 
complete  retest  involving  OT&E  Integration  and  Shakedown  tests 
was  scheduled  to  take  place  in  Retest  5.  No  deployment 
recommendation  was  made  by  ACW-200B. 

4.1.5  Retest  5. 

Retest  5  was  a  complete  test  of  the  RVR  designed  to  determine  if 
the  system  could  meet  minimum  operational  requirements  for  DVX. 

In  addition  to  verifying  the  resolution  of  problems  found  during 
previous  testing,  this  test  was  intended  to  obtain  optimum  values 
for  software  algorithm  parameters  under  actual  field  conditions. 
Refer  to  appendix  E  for  individual  test  reports. 
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4. 1.5.1  Background. 


Software  used  in  this  test  was  officially  released  after  the 
completion  of  factory  software  quality  control  tests.  As  in  the 
previous  test/  software  modifications  were  primarily  intended  to 
allow  the  system  to  be  used  at  the  DVX  which  was  scheduled  to 
open  in  December  of  1993. 

Prior  to  the  start  of  Retest  5,  performance  discrepancies  were 
noted  with  the  ALS  during  factory  qualification  tests.  The 
discrepancies  were  related  to  unexplained  window  signals  from  the 
ALS  when  no  apparent  stimulus  existed.  It  was  noted  that  these 
symptoms  were  exhibited  when  the  sensor  was  placed  in  sunlight. 
This  problem  was  not  corrected  prior  to  commencement  of  OT&E 
Integration  and  Shakedown  tests. 

In  conjunction  with  the  modifications  discussed  in  paragraph 
4. 1.4. 1.1  through  4. 1.4. 1.3,  several  significant  hardware  changes 
were  made  to  the  VS  and  ALS  (paragraph  1.3.5) .  These  hardware 
modifications  were  combined  with  software  changes  as  part  of  a 
•'rapid  prototyping"  effort  to  produce  an  RVR  system  for  use  at 
DVX. 


Due  to  limitations  in  equipment,  weather  conditions,  and  time, 
these  modifications  could  not  be  fully  evaluated  during  Retest  5. 
Specialized  tests  at  CRREL  and  Mt.  Washington  (paragraph  4.2.1, 
4.2.2)  were  established  to  conduct  these  evaluations.  These 
tests  were  conducted  concurrent  with  Retests  4  and  5  and  were 
developmental  in  nature. 

It  was  planned  to  retrofit  RVR  systems  installed  at  MCI  and  DVX 
following  successful  conclusion  of  specialized  tests  and 
implementation  of  final  sensor  design  changes. 

4. 1.5.2  Test  Date  and  Location. 

Testing  was  conducted  during  the  following  periods:  from 
September  20  through  Septeiaber  24,  1993,  at  MCI,  Kansas  City, 
Missouri.  OT&E  Shakedown  tests  were  conducted  from  September  20 
through  September  24,  and  OT&E  Integration  testing  was 
accomplished  from  September  22  through  September  23,  1993. 
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4. 1.5. 3  Participants. 


Personnel  from  the  following  organizations  conducted  and 
supported  OT&E  retest: 

Organization  Role 

Test  Director 
Test  Engineering 
Shakedown  Test  Engineering 
Observer 
Technicians 

Route  Traffic 

System  Administrators 

4. 1.5.4  Specialized  Test  Equipment. 

Test  equipment  included  protocol  analyzers  and  remote  maintenance 
simulation  software. 

4. 1.5. 5  Test  Objectives. 

Test  objectives  were  the  same  as  those  stated  in  paragraph 

4. 1.1. 5.  This  round  of  testing  was  intended  to  verify 
improvements  and  performance  in  the  areas  noted  in  paragraph 
4. 1.3. 8  and  4. 1.4. 8.  Also,  as  in  Retest  3,  test  objectives 
included  defining  the  extent  of  performance  degradation  expected 
as  a  result  of  deploying  a  system  with  known  discrepancies. 

4. 1.5. 6  Test  Descriptions. 

The  test  descriptions  were  the  same  as  stated  in  paragraph 

4. 1.1. 6. 

4. 1.5. 7  Data  Collection  and  Analysis. 

Test  conduct  forms  were  maintained  during  testing  and  discrepancy 
reports  were  created  as  issues  occurred.  Data  communication  log 
files  were  also  established  for  analysis  purposes. 

4 . 1 . 5 . 8  Test  Results . 

Test  results  indicated  the  further  corrections  within  the  RMM 
area.  RMS  interface  corrections  included  the  proper  prioritizing 
of  various  MPS  messages  emd  the  elimination  of  incomplete  MPS 
messages. 


ACW-200B 

ACN-IOOD 

AOS-220 

ANN-400 

MCI  AFSFO 

Kansas  City  Air 

Control  Center 
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Test  results  again  indicated  continuing  problems  in  the  following 
areas: 

a.  Sensor  performance, 

b.  System  documentation, 

c.  Maintenance, 

d.  Personnel  safety, 

e.  Remote  Maintenance  Monitoring,  and 

f.  Controller  Display. 

These  problem  areas  are  discussed  in  the  following  subparagraphs. 

4. 1.5. 8.1  Sensor  Performance. 

Contrary  to  expectations,  VS  failures  were  again  observed  during 
a  rain  event  and  during  simulated  rain  tests.  The  VS  receiver 
appeared  more  sensitive  to  the  effects  of  rain  on  the  sensor  lens 
than  the  transmitter. 

Unexplained  window  signals  were  observed  in  the  VS  when  exposed 
to  sunlight.  These  were  similar  to  those  observed  in  the  ALS 
during  factory  qualification  tests.  In  what  appeared  to  be  a 
related  problem,  it  was  noted  that  the  sensors  appeared 
particularly  sensitive  to  precipitation  in  the  presence  of 
sunlight.  This  sensitivity  resulted  in  sensor  failures  during 
testing. 

Although  VS  calibration  did  not  require  repeated  attempts  as  in 
the  previous  test  (paragraph  4. 1.4. 8),  the  calibration  plates 
were  not  within  the  tolerance  specified  by  the  Technical 
Instruction  book. 

It  was  also  noted  that  insect  repellent  paint,  used  on  the  VS  to 
prevent  spiders  from  nesting  in  the  sensor  hoods,  did  not  appear 
to  be  effective.  The  repellent  paint  was  one  of  several  methods 
attempted  in  an  effort  to  prevent  spiders  from  interfering  with 
sensor  operation. 

4. 1.5. 8. 2  System  Documentation. 

System  documentation  remained  as  stated  in  paragraph  4. 1.4. 8. 2. 

4. 1.5. 8. 3  Maintenance. 

Maintenance  problems  remained  as  stated  in  paragraph  4. 1.3. 8. 3. 
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4. 1.5. 8 .4  Remote  Maintenance  Monitoring. 


Corrections  to  RMM  discrepancies  were  limited  to  those  noted  in 
paragraph  4. 1.5. 8.  Other  previously  identified  problems 
remained.  For  the  first  time,  no  additional  problems  were  noted 
during  testing. 

4. 1.5. 8. 5  Personnel  Safety. 

Personnel  safety  problems  remained  as  stated  in  paragraph 

4. 1.1. 8. 5. 

4. 1.5. 8. 6  Controller  Display. 

CD  discrepancies  remained  as  stated  in  paragraph  4 . 1 . 3 . 8 . 6 . 

4 . 1 . 5 . 9  Recommendations . 

Continued  existence  of  sensor  problems  again  prevented 
consideration  of  the  RVR  system  for  national  deployment. 
Consideration  for  deployment  at  DVX  was  not  ruled  out,  because 
testing  of  the  Look-Down  VS  (paragraph  4.2.1)  at  CRREL  had  proven 
the  design  successful  in  preventing  difficulties  with 
precipitation . 

Based  on  test  results  at  MCI  and  CRREL,  the  test  organizations 
and  the  Program  Office  were  in  agreement  that  the  Look-Across  VS 
design  should  be  abandoned.  However,  because  the  software 
modifications  to  date  had  been  somewhat  successful,  the  decision 
was  made  to  continue  to  refine  the  contamination  algorithms.  The 
software  modifications  would  be  the  only  means  of  desensitizing 
the  ALS  with  respect  to  precipitation  related  shutdowns. 

4.1.6  Retest  6 . 

Retest  6  was  intended  to  qualify  an  interim  version  of  RVR 
software  for  deployment  at  DVX.  It  was  also  intended  to  be  a 
full  retest  and  as  such,  satisfy  part  of  the  conditions  required 
for  an  operationally  acceptable  system.  Refer  to  appendix  F  for 
individual  test  reports. 

4 . 1 . 6 . 1  Background . 

In  the  period  between  Retest  5  and  Retest  6,  results  of 
specialized  testing  were  discussed  at  several  test  assessment  and 
design  review  meetings.  Results  indicated  that  major  problems 
and  areas  of.  uncertainty  were  very  close  to  being  resolved.  In 
anticipation  of  a  deployment  decision  for  both  DVX  and  national 
deployment,  discussions  focused  on  determining  what  corrections 
and  modifications  to  the  system  had  to  be  done  predeployment,  and 
which  could  be  accomplished  post-deployment. 
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Retest  6  included  production  release  software  and  new  prototypes 
of  the  CD,  ALS,  and  Look-Down  VS.  In  addition,  modifications  to 
the  CD  were  incorporated  in  response  to  previous  comments  from 
air  traffic  controllers.  As  in  the  previous  test,  this  version 
of  hardware  and  software  was  designated  as  the  Denver 
Conf igurat ion . 

4. 1.6. 2  Test  Date  and  Location. 

Testing  was  conducted  from  December  6  through  December  10,  1993, 
at  MCI,  Kansas  City,  Missouri.  OT&E  Integration  and  Shakedown 
tests  were  conducted  using  day  and  evening  shifts  during  this 
period . 


4. 1.6. 3  Participants. 


Personnel  from  the  following  organizations  conducted  and 
supported  OT&E  retest: 


Organization 

ACW-200B 

ACN-IOOD 

AOS-220 

ANN-400 

MCI  Air  Traffic  Controllers 
MCI  AFSFO 

Kansas  City  Air  Route  Traffic 
Control  Center 


Role 

Test  Director 
Test  Engineering 
Shakedown  Test  Engineering 
Observer 

User  Evaluations 
Technicians 

System  Administrators 


4. 1.6. 4  Specialized  Test  Equipment. 

Test  eguipment  included  protocol  analyzers  and  remote  maintenance 
simulation  software. 


4. 1.6. 5  Test  Obiectives. 

In  addition  to  fulfilling  those  stated  in  paragraph  4. 1.1. 5,  test 
objectives  included  assessing  improvements  to  problems  as  stated 
in  paragraph  4. 1.5. 8  and  conducting  a  performance  evaluation  of 
the  newly  installed  Look-Down  VS.  An  additional  objective  was  to 
have  air  traffic  controllers  evaluate  the  new  CD  prototype. 

4. 1.6. 6  Test  Descriptions. 

Because  this  was  considered  to  be  a  predeployment  test  for  DVX, 
the  number  of  procedures  conducted  was  expanded  to  ensure  a 
thorough  exercise  of  system  functionality. 
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4. 1.6. 7  Data  Collection  and  Analysis. 


Test  conduct  forms  were  maintained  during  testing  and  discrepancy 
reports  were  created  as  issues  occurred.  Data  communication  log 
files  were  also  established  for  analysis  purposes. 

4. 1.6. 8  Test  Results. 


As  opposed  to  Retest  5,  calibration  of  the  VS  was  achieved 
without  repeated  attempts.  Sensor  calibration  plates  were  within 
the  tolerance  specified  by  the  Technical  Instruction  manual. 

Problems  continued  in  the  areas  shown  below: 

a.  Sensor  performance, 

b .  System  documentation , 

c.  Maintenance, 

d.  Personnel  safety, 

e.  Remote  Maintenance  Monitoring,  and 

f.  Controller  Display. 

These  problem  areas  are  discussed  in  the  following  subparagraphs. 
4 . 1 . 6 . 8 . 1  Sensor  Performance . 

This  was  the  first  retest  performed  with  the  look-down 
configuration  VS.  Performance  discrepancies  related  to  RVR 
product  calculations  and  sensor  shutdowns  were  still  observed 
during  test  scenarios  using  simulated  precipitation. 

It  was  noted,  however,  that  the  VS  seemed  more  sensitive  to  water 
falling  through  the  sample  volume  than  actually  hitting  the 
sensor  lens.  Testing  proved  this  phenomena  to  be  directly 
related  to  the  presence  of  sunlight  during  the  simulated 
precipitation.  The  problem  was  later  attributed  to  an  inability 
of  the  sensor  to  compensate  for  dynamic  components  of  sunlight. 

Discrepancies  with  ALS  window  contamination  readings  were  noted 
during  simulated  precipitation.  The  discrepancies  were 
determined  to  be  caused  by  an  incorrect  contamination  gain  value. 
This  result  indicated  the  need  for  further  evaluation  and 
adjustment  of  the  contamination  gain  value. 

It  was  noted  by  the  test  team  that  it  was  difficult  to  evaluate 
proper  operation  of  the  contamination  alarms  because  there  was  no 
indication  as  to  what  mode  (e.g.,  precipitation,  dirt)  the  sensor 
was  in . 
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4. 1.6. 8. 2  System  Documentation. 

Although  significant  updates  to  the  On-Site  Technical  Instruction 
manual  were  previously  incorporated,  additional  errors  were  found 
in  the  manual.  Correction  of  the  errors  was  recommended  prior  to 
delivery  of  the  manual  to  DVX. 

4. 1.6. 8. 3  Maintenance. 

Continued  existence  of  maintenance  problems  and  issues  was 
observed  during  testing.  In  particular,  the  inability  of  the  SIE 
to  be  reset  while  on  battery  power  was  noted.  This  problem  was 
determined  to  be  related  to  a  previously  identified  problem 
involving  a  lack  of  communication  between  the  SIE  and  the  DPU 
when  the  SIE  operated  on  battery  power. 

4 . 1 . 6 . 8 . 4  Remote  Maintenance  Monitoring . 

Previously  identified  problems  as  stated  in  paragraph  4. 1.3. 8. 4 
remained. 

4. 1.6. 8. 5  Personnel  Safety. 

Previously  observed  personnel  safety  problems  remained  as  stated 
in  paragraph  4 . 1 . 1 . 1 . 5 . 

4. 1.6. 8. 6  Controller  Display. 

MCI  Airport  air  traffic  controllers  conducted  a  daytime 
evaluation  of  the  RVR  CD  at  the  MCI  TRACON  and  the  control  tower 
cab.  In  general,  controllers  were  pleased  with  the  changes 
incorporated . 

Controllers  recommended  the  following  minor  changes: 

a.  The  ability  to  adjust  brightness  of  the  power  switch 
and  the  Health  LED;  and 

b.  Separate  backlighting  controls  for  the  product  display 
and  keypad. 

The  Test  Director  noted  concerns  that  the  evaluation  was  not 
conducted  under  a  variety  of  lighting  conditions. 


38 


4. 1.6.9  Recommendations. 


Shutdowns  were  much  more  difficult  to  produce,  and  those  observed 
were  short-lived.  In  addition,  shutdowns  now  appeared  to  be 
primarily  related  to  the  presence  of  sunlight  during 
precipitation.  Although  any  occurrence  of  shutdowns  was 
considered  unacceptable,  the  likelihood  of  operational  impacts 
caused  by  sunlight— related  shutdowns  was  considered  to  be  remote. 

Retest  6  results  provided  the  basis  for  a  limited  deployment 
recommendation  for  DVX.  The  system  and  ongoing  activities  met 
the  criteria  for  minimum  "operational  acceptability"  as  defined 
in  paragraph  4 . 1 . 4 . 1 . 

Justification  was  as  follows: 

a.  Test  results  indicated  no  new  significant  problem  areas 
existed; 

b.  Work  was  started  on  a  notice,  INTERIM  CERTIFICATION  AND 
MAINTENANCE  OF  THE  RUNWAY  VISUAL  RANGE  SYSTEM,  TYPE  FA- 
10268,  N  6560.16  detailing  work-around  procedures  and 
existing  problems  that  should  be  monitored  by 
maintenance  technicians.  This  notice  was  planned  for 
distribution  to  appropriate  personnel  at  DVX; 

c.  Activities  for  resolving  existing  problem  areas  were 
on-going  and  were  expected  to  resolve  remaining 
significant  issues  before  national  deployment  of  the 
RVR;  and 

d.  A  complete  retest  of  the  RVR  system  was  accomplished 
during  this  time  period. 

Further  evaluation  and  adjustment  of  the  contamination  gain  value 
parameter,  and  the  correction  of  errors  to  the  On-Site  Technical 
Instruction  manual  were  recommended. 

Additionally,  since  new  modes  of  operation  (e.g.,  rain  and  dirt) 
were  incorporated  in  the  VS  and  ALS,  it  was  recommended  that  an 
indication  of  these  modes  be  incorporated  in  the  system. 

Based  on  the  above  test  results,  ACW-200B  continued  to  consider 
the  RVR  system  not  ready  for  national  deployment.  Requirements 
for  national  deployment  included  additional  blowing  precipitation 
tests  along  with  a  more  thorough  assessment  of  system  performance 
and  accuracy  during  Category  mb  visibility. 
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4 ♦ 1 . 7  Retest  7 . 


Results  of  specialized  testing,  along  with  improvements  resulting 
from  additional  system  modifications,  indicated  that  the  RVR  was 
finally  ready  for  national  deployment.  In  preparation  for  this 
final  retest,  RVR  hardware  and  software  were  officially  released 
as  First  Articles  after  completing  factory  qualification  tests. 
Retest  7  was  expected  to  be  the  final  test  of  the  RVR  system 
prior  to  a  deployment  decision.  Refer  to  appendix  G  for 
individual  test  reports. 

4 . 1 . 7 . 1  Background . 

As  mentioned  in  paragraph  4. 1.6.1,  RVR  modifications  in  response 
to  noted  problems  were  planned  for  predeployment  as  well  as  post¬ 
deployment.  Retest  7  marked  the  conclusion  of  planned 
predeployment  corrections. 

Incorporated  improvements  are  listed  below: 

SIE 

a.  Increase  corrosion  resistance  of  enclosures; 

b.  Increase  accessibility  of  internal  components; 

c.  Reduce  safety  hazards  during  maintenance; 

d.  Add  needed  MDT  maintenance  parameters;  and 

e.  Increase  Electromagnetic  Interference  resistance; 

CD 

a.  Improve  daytime/nighttime  readability; 

b.  Incorporate  data  loss  protection;  and 

c.  Add  software  expansion  capabilities; 

VS 

a.  Reduce  potential  of  ice/ snow  clogging; 

b.  Reduce  noise  in  ambient  light  readings;  and 

c.  Reduce  potential  of  sensor  failures  resulting  from 
window  contamination. 

ALS 


a.  Reduce  potential  of  sensor  failures  resulting  from 
window  contamination;  and 

b.  Reduce  potential  of  snow/ ice  accumulation. 
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DPU 


a.  Elimination  of  failures  during  communication  with 
subsystems ; 

b.  Complete  remote  and  local  maintenance  capabilities;  and 

c.  Reduce  potential  for  safety  hazards  during  maintenance. 

In  preparation  for  post-deployment  modifications,  interim  design 
changes  were  implemented  into  the  system.  These  included 
additional  control  cables,  connectors,  and  thermal  sensors  for 
the  VS  and  ALS.  The  addition  of  these  components  was  part  of  an 
effort  to  further  reduce  the  potential  for  snow  and  ice  clogging 
by  using  a  more  complex  heater  control  scheme.  The  complete 
functionality  of  the  modified  scheme  will  be  realized  when  post¬ 
deployment  modifications  are  completed. 

VS  accuracy  and  performance  data  were  obtained  from  data 
collection  sites  during  the  period  between  Retest  6  and  Retest  7. 
As  mentioned  in  paragraph  1.3. 3.1,  these  locations  were 
established  to  collect  sufficient  amounts  of  data  on  VS 
performance  under  actual  blowing  precipitation  conditions  and 
fog. 

Data  from  the  collection  sites,  as  well  as  results  of  specialized 
tests,  indicated  that  the  look-down  configuration  VS  was 
extremely  effective  in  reducing  sensor  failures  due  to  snow  and 
ice  clogging.  The  look-down  design  also  eliminated  large  window 
signals  caused  by  precipitation  striking  the  VS  lens. 

Reduced  visibility  testing  indicated  that  this  configuration  was 
capable  of  operation  within  tolerance  during  actual  Category  Illb 
conditions  (paragraph  4.2.2.11). 

4. 1.7. 2  Test  Date  and  Location. 

Testing  was  conducted  during  the  following  periods:  June  6 
through  June  10,  1994  (Shakedown) ,  and  June  13  through  June  20 
(Integration)  at  MCI,  Kansas  City,  Missouri. 
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4. 1.7. 3  Participants. 


Personnel  from  the  following  organizations  conducted  and 
supported  OT&E  retest: 


Organization 


Role 


ACW-200B 

ACN-IOOD 

AOS-220 

ANN-400 

MCI  Air  Traffic  Controllers 


MCI  AFSFO 

Kansas  City  Air  Route  Traffic 
Control  Center 


Test  Director 

Test  Engineering 

Shakedown  Test  Engineering 

Observer 

User  Evaluation 

Technicians 

System  Administrators 


4. 1.7. 4  Specialized  Test  Equipment. 

Test  equipment  included  protocol  analyzers  and  remote  maintenance 
simulation  software. 


4. 1.7. 5  Test  Objectives. 

Test  efforts  were  focused  on  an  in-depth  evaluation  of  the  First 
Article  components.  The  primary  objective  was  to  ensure  that  all 
predeployment  modifications  had  been  completed  and  were  working 
as  intended. 


4. 1.7. 6  Test  Descriptions. 

The  test  descriptions  were  the  same  as  stated  in  paragraph 

4. 1.6. 6. 

4. 1.7.7  Data  Collection  and  Analysis. 

Test  conduct  forms  were  maintained  during  testing  and  discrepancy 
reports  were  created  as  issues  occurred.  Data  communication  log 
files  were  also  established  for  analysis  purposes. 

4. 1.7. 8  Test  Results. 

Test  results  indicated  resolution  of  nearly  all  items  identified 
as  predeployment  critical. 

Work-around  solutions  were  implemented  or  planned  for  problems 
which  could  not  be  totally  resolved  before  national  deployment. 

A  discussion  of  results  is  provided  in  the  following 
subparagraphs . 
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4. 1.7. 8.1  Sensor  Performance. 


Modifications  to  ALS  window  contamination  algorithms  were 
successful  in  preventing  failures  during  simulated  precipitation 
tests.  Modifications  also  resulted  in  window  contamination 
readings  which  appeared  to  be  accurate  throughout  test  scenarios. 

In  general,  VS  were  more  resistant  to  failures  occurring  from 
precipitation  striking  the  receiver  and  transmitter  lenses  than 
in  previous  tests.  When  failures  occurred,  recovery  time  was 
significantly  shorter  than  in  previous  tests.  It  should  be  noted 
that  the  likelihood  of  precipitation  striking  the  VS  lenses 
during  typical  rain  or  snow  storms  was  greatly  reduced  by  the 
look-down  design. 

Despite  addition  of  optical  filters,  erratic  performance  was 
again  observed  during  simulated  precipitation  with  bright 
sunlight.  However,  due  to  the  infrequent  occurrence  of 
precipitation  with  bright  sunlight  and  the  high  runway  visibility 
associated  with  these  conditions,  this  problem  was  considered 
noncritical. 

Difficulties  were  again  encountered  with  VS  calibration;  however, 
it  appeared  that  the  problem  was  related  to  cloudy  weather 
conditions.  Although  this  problem  was  considered  noncritical, 
ACW-200  recommended  an  investigation  into  the  cause  followed  by  a 
post-deployment  resolution. 

4. 1.7. 8. 2  System  Documentation. 

The  On-Site  Technical  Instruction  manual  contained  nearly  all  of 
the  updates  requested  by  the  test  organizations.  Some  procedures 
and  chapters  were  still  deficient  in  content  or  contained  errors. 
In  particular,  the  battery  checkout  procedure  was  in  error  and 
did  not  reflect  differences  between  VS  and  ALS  SIEs.  Additional 
updates  and  correction  of  noted  errors  were  recommended  for  the 
On-Site  Technical  Instruction  manual . 

4. 1.7. 8. 3  Maintenance. 

Most  maintenance  problems  were  resolved  with  the  advent  of  First 
Article  SIEs,  DPU,  and  maintenance  procedures.  In  particular, 
problems  noted  with  SIE  corrosion  and  lightning  protection  were 
resolved.  Battery  monitoring  problems  were  resolved  by 
incorporating  additional  monitoring  capabilities  into  the  SIEs 
and  by  adding  maintenance  procedures  to  eliminate  the  possibility 
of  undetected  battery  failures. 

Maintenance  displays  on  the  MDT  were  modified  to  allow  reading 
and  setting  of  RVR  parameters  without  using  conversion  factors  to 
change  numbers  to  common  units. 


43 


Outstanding  problems  included  the  inability  of  the  RLIM  to  detect 
sensor  faults.  The  solution  for. this  problem  was  an  addition  to 
the  On-Site  manual  notifying  airport  technicians  of  the  sensor 
limitations.  All  other  maintenance  problems  were  considered  non- 
critical. 

4. 1.7. 8. 4  Remote  Maintenance  Monitoring. 

Test  results  indicated  the  correction  of  most  RMM  problems 
described  previously.  In  particular,  significant  problems  with 
failures  in  alarm  generation  or  clearing  were  corrected.  Also, 
problems  including  RMS  message  formatting  and  the  execution  of 
fault  diagnostic  commands  were  corrected. 

Significant  in  the  outstanding  problems  was  the  occurrence  of 
communication  failures  between  the  RVR  RMS  and  the  MPS.  Effects 
of  this  problem  include  the  failure  of  the  RVR  RMS  to  send 
priority  messages  to  the  MPS.  This  had  been  noted  in  previous 
test  results,  but  was  not  resolved  due  to  difficulties  in 
consistently  reproducing  the  problem.  It  was  expected  that  an 
automated  work-around  solution  could  be  implemented  after 
testing.  ACW-2 00  requested  the  work-around  be  developed 
immediately  and  be  implemented  in  any  system  scheduled  for 
commissioning.  No  other  significant  problems  were  noted  during 
testing. 

4. 1.7. 8. 5  Personnel  Safety. 

Nearly  all  personnel  safety  issues  were  resolved.  Electrical 
shock  hazards  were  reduced  by  moving  voltage  terminals  and  adding 
terminal  block  covers.  Warning  labels  were  also  added. 

The  potential  for  physical  injury  while  raising  or  lowering  VS 
poles  continued  to  be  an  issue.  A  modification  to  the  mechanical 
lift  hardware  was  recommended  to  resolve  this  problem. 

4. 1.7. 8. 6  Controller  Display. 

The  ability  to  adjust  display  and  function  key  brightness  was 
incorporated.  Data  loss  protection  and  additional  memory  were 
added . 

4 . 1 . 7 . 9  Recommendations . 

In  light  of  the  resolution  of  the  more  serious  system 
discrepancies,  as  well  as  many  of  the  less  significant  problems, 
a  national  deployment  recommendation  was  issued  by  ACW-200B. 

This  recommendation  was  subject  to  the  following  conditions: 

a.  The  immediate  investigation  and  implementation  of  an 
automated  work-around  solution  for  the  communication 
problem  involving  the  RVR  RMS  interface; 
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b.  The  implementation  of  a  final  solution  to  the  RMS 
interface  communication  problem  after  national 
deployment; 

c.  Additional  testing  to  better  define  the  response  of  the 
system  to  precipitation  during  sunlight; 

d.  Collection  of  additional  performance  data  under 
Category  Illb  conditions;  and 

e.  Investigate  and  correct  calibration  difficulties 
related  to  cloudy  weather  conditions. 

4.2  SPECIALIZED  TESTS. 

Specialized  testing  of  RVR  components  began  following  Retest  3. 
The  lack  of  any  real  progress  in  eliminating  weather  related 
sensor  problems  dictated  the  need  to  initiate  a  more  focused 
development  and  test  effort  in  this  area.  The  need  was 
intensified  by  pressures  to  field  an  acceptable  system  at  DVX, 
even  though  a  national  deployment  system  was  still  considered  to 
be  months  away.  The  specialized  test  effort  was  conducted  on  an 
accelerated  schedule  which  ran  concurrent  with  the  schedule  for 
Retests  4  and  5. 

4.2.1  Cold  Regions  Research  and  Engineering  Laboratory  Tests . 

CRREL  tests  were  intended  to  assess  the  performance  of  the  VS  and 
ALS  under  blowing  precipitation  and  Category  Illb  conditions. 
These  tests  were  conducted  during  the  same  time  period  as  Retest 
4.  Refer  to  appendix  H  for  detailed  reports  concerning  test 
activities  at  CRREL. 

4 . 2 . 1 . 1  Background . 

The  CRREL  is  a  facility  owned  by  the  U.S.  Army.  It  has  the 
capability  to  provide  simulations  of  cold  weather  phenomenon 
including  below  zero  temperatures,  snow,  and  freezing  rain.  The 
laboratory  includes  various  temperature  chamber  rooms  and 
apparatus  for  reproducing  diverse  weather  conditions. 

Deficient  performance  during  blowing  precipitation  was  known  to 
be  a  problem  throughout  RVR  test  history.  Modifications  such  as 
the  addition  of  precipitation  filters  and  enhanced  precipitation 
algorithms  were  unsuccessful  in  preventing  serious  problems 
including  sensor  failures,  erratic  visibility  readings,  and 
system  shutdowns. 
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4.2. 1.4  Specialized  Test  Equipment. 

Various  equipment  was  used  in  the  simulation  of  blowing 
precipitation  conditions  and  for  data  collection.  A  listing  of 
the  apparatus  used  in  the  simulation  of  blowing  precipitation  are 
provided  as  follows: 

a.  Temperature  chambers, 

b.  Wind  tunnels, 

c.  Snow  generation  apparatus, 

d.  Mist  generators,  and 

e .  Fog  generators . 

Data  collection  devices  included  the  following: 

a.  Data  acquisition  computers, 

b.  Infrared  video  cameras,  and 

c .  Video  cameras . 

4.2. 1.5  Test  Objectives. 

In  general,  the  objectives  of  testing  were  to  find  ways  to 
improve  VS  and  ALS  performance  under  blowing  precipitation  and  to 
acquire  data  to  verifying  Category  Illb  performance. 

Because  the  RVR  system  had  consistently  shown  weaknesses  in 
performance  during  blowing  precipitation,  CRREL  tests  were 
designed  to: 

a.  Assess  severe  weather  performance  under  controlled 

conditions ; 

b.  Provide  data  for  development  of  system  modifications; 

c.  Provide  direct  comparisons  between  different 
configurations  of  VS  and  ALS; 

d.  Determine  optimum  sensor  design;  and 

e.  Gain  a  better  understanding  of  the  effects  of  blowing 
precipitation;  particularly,  snow  and  ice  clogging, 
high  window  signal  levels,  and  extinction  coefficient 
readings . 

4 . 2 . 1 . 6  System  Conf iourat ion . 

Because  CRREL  tests  were  intended  primarily  to  evaluate  the  VS 
and  ALS,  a  complete  system  configuration  was  not  required.  As  a 
result,  RLIMs,  CDs,  and  the  RMS  interface  were  not  used.  The 
exclusion  of  these  system  components  had  no  effect  on  test 
results . 
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For  the  CRREL  tests,  the  RVR  system  consisted  of  the  following 
components  and  interfaces: 

Components 

a.  Visibility  Sensors  (2)  .  One  look-down  configuration 
and  one  look-across  configuration,  installed  inside  of 
test  chamber; 

b.  Ambient  Light  Sensor  (1).  Installed  inside  of  test 
chamber ; 

c.  Data  Processing  Unit  (1) .  Installed  outside  of  test 
chamber ;  and 

d.  Sensor  Interface  Electronics  enclosures  (3).  Installed 
inside  of  test  chamber. 


Interfaces 


a.  Maintenance  Data  Terminal.  The  MDT  interface  was  used 
to  monitor  system  parameters  during  testing;  and 

b.  External  User.  The  EU  interface  was  used  throughout 
testing  for  the  collection  of  system  parameter  data. 

4 . 2 . 1 . 7  Test  Descriptions . 

The  tests  performed  fell  under  one  of  two  main  categories  of 
evaluations;  blowing  precipitation  or  low  visibility  (fog) . 
Descriptions  are  provided  in  the  following  subparagraphs. 

4. 2. 1.7.1  Blowing  Precipitation. 

Blowing  precipitation  tests  consisted  of  various  scenarios  of 
blowing  snow,  rain,  and  mist  directed  at  the  sensors  for 
sustained  intervals.  As  weaknesses  or  trends  were  observed  in 
sensor  performance,  additional  test  scenarios  were  created  to 
obtain  more  information.  For  example,  if  it  appeared  that  snow 
accumulation  or  high  window  contamination  readings  were  dependent 
on  the  direction  of  precipitation,  additional  test  scenarios 
would  be  created  to  determine  which  directions  caused  problems. 

Any  weaknesses  observed  were  noted  by  the  test  team.  Information 
was  collected  via  video  film  recordings,  DPU  monitoring,  and 
written  observations.  Included  in  the  assessment  were  direct 
comparisons  between  the  look-down  and  look-across  configurations 
under  identical  test  conditions. 
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Effects  of  blowing  precipitation,  such  as  snow/ ice  accumulation, 
window  signal  patterns,  and  extinction  coefficient  fluctuation, 
were  observed  and  noted  by  the  test  team.  Testing  was  designed 
to  determine  the  limits  of  sensor  tolerance  to  blowing 
precipitation  for  each  of  the  sensor  prototypes. 

4. 2. 1.7. 2  Low  visibility  Tests. 

To  assess  Category  IXlb  performance,  fog  was  simulated  with  test 
equipment.  .Testing  consisted  of  comparing  RVR  readings  with  an 
Optec  transmissometer  under  the  same  Category  Illb  conditions. 

The  Optec  was  modified  to  operate  over  a  short  baseline  to 
accurately  measure  Category  Illb  visibility. 

4. 2. 1.8  Data  Collection  and  Analysis. 

Extensive  data  were  collected  in  the  form  of  computer  and  video 
files  as  well  as  observation  notes.  Data  analysis  consisted  of  a 
review  of  sensor  performance  after  each  test  scenario. 

4 . 2 . 1 . 9  Test  Results . 

Results  of  blowing  precipitation  and  low  visibility  tests  are 
summarized  in  the  following  subparagraphs. 

4 . 2 . 1 . 9 . 1  Blowing  Precipitation . 

Prototype  VS  and  ALS  designs  exhibiting  improved  blowing 
precipitation  performance  were  established.  The  design 
improvements  were  successful  in  reducing  the  probability  of 
sensor  failures;  and  hence,  system  shutdowns  during  test 
conditions. 

Design  prototypes  included  the  Look-Down  VS  with  extended  hood 
heaters;  and  an  ALS  with  a  higher  power  and  more  Conformal  hood 
heater  design.  Results  also  prompted  the  removal  of  a  mounting 
ledge  on  the  ALS  which  significantly  reduced  its  susceptibility 
to  snow  clogging. 

4. 2. 1.9. 2  Low  Visibility  Tests. 

A  significant  amount  of  Category  Illb  data  was  obtained;  however, 
due  to  problems  in  sustaining  Category  Illb  conditions,  as  well 
as  erratic  and  uncorrelated  VS  readings.  Category  Illb  test 
results  were  considered  inconclusive. 
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4.2.1.10  Recommendations. 


Based  on  test  results,  additional  modifications  were  recommended 
by  the  test  team.  The  recommended  modifications  included 
additional  changes  to  the  hood  and  window  heater  designs  of  the 
Look-Down  VS  and  the  ALS.  The  addition  of  more  sophisticated 
heater  control  schemes  to  further  reduce  the  possibility  of  ice 
or  snow  accumulation  was  also  recommended. 

Although  it  appeared  that  a  significant  improvement  in 
performance  could  be  attained  with  the  new  sensor  designs,  the 
acquisition  of  test  data  during  actual  blowing  precipitation 
conditions  was  strongly  recommended  by  the  test  team;  hence,  it 
was  recommended  that  data  be  collected  from  the  automated  data 
collection  sites  identified  in  paragraph  1.3. 3.1. 

4.2.2  Category  Illb  Validation. 

This  section  describes  results  of  Category  Illb  testing  performed 
on  the  RVR  system.  The  intent  of  the  tests  was  to  provide  RVR 
performance  data  during  actual  Category  Illb  visibility 
conditions.  Refer  to  appendix  I  for  individual  reports. 

4 . 2 . 2 . 1  Background . 

Although  it  had  been  shown,  in  theory,  that  the  New  Generation 
RVR  could  perform  within  specification  throughout  the  Category 
Illb  range,  little  data  existed  for  performance  under  actual 
conditions.  This  was  the  first  evaluation  of  the  RVR  conducted 
specifically  to  evaluate  performance  under  actual  Category  Illb 
weather  conditions . 

The  Category  Illb  tests  were  part  of  the  rapid-prototyping  effort 
started  with  the  CRREL  tests.  As  part  of  this  effort,  the  look- 
down  and  look-across  configuration  VS  were  compared  for  accuracy, 
and  resistance  to  extreme  weather  conditions.  Category  mb 
tests  were  conducted  during  the  same  timeframe  as  Retest  5. 

4. 2. 2. 2  Test  Date  and  Location. 

Testing  was  conducted  from  September  13  through  21,  1993,  at  the 
summit  of  Mt.  Washington  in  New  Hampshire. 
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4. 2. 2. 3  Participants. 


Personnel  from  the  following  organizations  conducted  and 
supported  this  test: 


Organization 

VNTSC 

AOS-220 

ACW-200B 

Mt .  Washington  Observatory 

4. 2. 2. 4  Specialized  Test  Equipment. 


Role 


Test  Engineering 
Test  Engineering 
Test  Director 
Resource  Support 


Specialized  equipment  used  during  testing  is  listed  below.  The 
purpose  for  each  device  is  briefly  described  in  subsequent 
paragraphs . 

a.  Runway  centerline  light  fixture, 

b.  Variac  power  supply, 

c.  Rackmount  data  acquisition  computer, 

d.  Optec  Long-Path  Visibility  Transmissometer ,  and 

e.  Portable  PC  with  RVR  measurement  software. 

4 . 2 . 2 . 4 ♦ 1  Runway  Centerline  Light  Fixture . 


A  centerline  light  fixture,  identical  to  those  used  at  most  U.S. 
airports,  was  used  as  a  high-intensity  target  to  measure  viewing 
distances  by  test  team  observers. 


4 . 2 . 2 . 4 . 2  Variac  Power  Supply . 

A  variac  power  supply  was  used  to  provide  the  required  power  to 
the  runway  light  fixture  for  operation  at  the  standard  light 
settings  used  in  airports.  Standard  runway  light  settings  one 
through  five  were  used  during  testing. 

4. 2. 2. 4. 3  Rackmount  Data  Acquisition  Computer. 


A  rackmount  data  acquisition  computer  was  used  to  record  RVR  and 
transmissometer  data  during  testing.  This  information  was 
accessed  via  the  RVR  EU  port  and  a  data  port  on  the  Optec 
Transmissometer . 


4. 2. 2. 4. 4  Qptec  Long-Path  Visibility  Transmissometer. 

The  Optec  Transmissometer  was  used  as  a  reference  for  comparing 
RVR  readings  during  test  scenarios.  Because  this  transmissometer 
is  normally  used  to  measure  distances  greater  than  those  in  the 
Category  Illb  range,  it  was  modified  (the  baseline  was  shortened) 
to  measure  distances  required  during  testing. 
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4. 2. 2. 4. 5  Laptop  PC  and  RVR  Measurement  Software. 

A  laptop  PC  and  RVR  measurement  software  were  used  to  facilitate 
the  process  of  conducting  observations  by  the  test  team.  The  RVR 
measurement  software  was  a  duplication  of  the  algorithm  used  in 
the  RVR  to  calculate  runway  visual  range.  It  was  designed  to 
utilize  sensor  data  from  the  EU  port  to  provide  the  RVR  normally 
displayed  at  the  CD.  Using  the  PC  and  RVR  measurement  software, 
the  test  team  was  able  to  record  human  observations  and  RVR 
system  output  simultaneously. 

4. 2. 2. 5  Test  Objectives. 

The  primary  objective  was  to  provide  data  reflecting  RVR 
performance  during  the  full  range  of  Category  Illb  conditions. 

Secondary  objectives  included: 

a.  Comparing  RVR  readings  from  the  Look-Down  and  Look- 
Across  VS  to  identify  any  inconsistencies  between  the 
designs;  and 

b.  Noting  any  significant  effects  of  weather  on  VS 
measurements . 

4. 2. 2. 6  System  Configuration. 

Category  Illb  tests  were  evaluations  of  sensor  performance  as 
opposed  to  system  performance;  consequently,  RLIMs  and  CDs  were 
not  required.  RLIM  values  for  calculating  RVR  were  input 
manually  by  the  test  team. 

The  test  configuration  was  as  shown  below; 

Hardware 

a.  Visibility  Sensors  (3).  One  look-down  and  two  look- 
across  were  installed  on  12— foot  poles  at  the  mountain 
summit ; 

b.  Ambient  Light  Sensor  (1).  Installed  on  a  12-foot  pole 
at  the  mountain  summit; 

c.  Data  Processing  Unit  (1) .  Installed  inside  the  Mt. 
Washington  Observatory;  and 

d.  Sensor  Interface  Electronics  enclosures  (4). 

Collocated  with  each  sensor. 
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Interfaces 


a.  Maintenance  Data  Terminal.  The  MDT  interface  was  used 
to  monitor  system  parameters  during  testing;  and 

b.  External  User.  The  EU  interface  was  used  throughout 
testing  for  the  collection  of  system  parameter  data. 

4. 2. 2. 7  Test  Descriptions. 

Testing  consisted  of  making  and  comparing  the  visibility 
measurements  of  a  human  observer,  a  transmissometer ,  and  the  RVR 
VS. 


Refer  to  the  "Category  Illb  test  report  for  the  New  Generation 
Runway  Visual  Range"  in  appendix  I  for  a  detailed  description  of 
test  procedures. 

4. 2 .2. 8  Data  Collection  and  Analysis. 

Data  was  collected  via  the  data  acquisition  system  and  notes 
taken  by  the  test  team.  Analysis  was  conducted  throughout  the 
testing  period.  Graphs  were  developed  showing  the  relationship 
of  visibility  measurements  of  the  RVR  sensors,  the  Optec 
Transmissometer,  and  human  observers. 

4. 2. 2. 9  Test  Limitations. 

Because  the  Category  Illb  testing  was  conducted  under  real 
weather  conditions,  scientific  controls  were  difficult  to 
implement  and  limitations  were  inherent  in  the  procedures  and 
equipment.  These  limitations  are  discussed  in  the  following 
subparagraphs . 

4. 2. 2. 9.1  Limitations  in  Test  Equipment. 

The  RVR  system  calculations  assume  the  use  of  an  aged,  dirty 
runway  centerline  light.  The  light  used  for  testing  was  new  and 
clean. 


4. 2. 2. 9. 2  Limitations  in  Test  Setup. 

In  general,  the  test  setup  was  a  simplified  approach  for 
measuring  runway  visibility.  The  amount  of  space  available  for 
use  at  the  summit  of  Mt.  Washington  was  limited.  As  a  result, 
the  entire  Category  Illb  range  could  not  be  tested. 

In  addition,  the  sensors  and  the  observers  were  not  collocated 
for  most  of  the  observations;  equal  fog  densities  at  both 
locations  could  not  be  guaranteed. 
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Lastly,  for  short  observation  distances  (e.g.,  less  than  100 
feet) ,  observers  were  outside  the  primary  path  of  the  runway 
light  beam.  Visibility  conditions  during  the  test  period 
provided  very  few  observations  in  this  range. 

4. 2. 2. 9. 3  Limitations  in  Test  Conduct. 

For  some  daylight  conditions,  RVR  calculations  are  based  on  a 
black  target  instead  of  a  runway  light.  This  is  done  because, 
under  certain  lighting  conditions,  the  black  target  presents  more 
contrast  than  the  runway  light.  For  the  Category  Illb  tests, 
observers  used  only  the  runway  centerline  light. 

4. 2. 2. 9. 4  Limitations  in  Analysis. 

No  accepted  standard  exists  for  RVR  measurement.  Therefore, 
performance  evaluation  was  done  by  cross  comparison  of  different 
methods  of  measuring  RVR.  Methods  of  measurement  for  this 
testing  included  the  New  Generation  RVR,  the  Optec  j  j 

Transmissometer,  and  human  observers.  This  lack  of  a  standard, 
as  well  as  the  use  of  human  observers,  leads  to  an  inherent 
subjectivity  in  test  results. 

4.2.2.10  Test  Results. 

Test  results  indicated  that  the  average  difference  between  RVR 
readings  and  observed  visibility  was  within  100  feet.  One 
hundred  feet  equates  to  one  reporting  unit  of  the  RVR  system  and 
is  within  the  requirements  of  the  RVR  specification.  It  should 
be  noted  that  although  the  average  measurements  were  within 
specification,  some  out-of-spec  measurements  were  recorded.  See 
appendix  I  for  details  on  measurements  and  possible  rationale  of 
out-of-spec  data. 

Test  results  also  indicated  that,  due  to  its  inability  to  perfora 
during  severe  weather  conditions  at  Mt.  Washington,  data  from  the 
Optec  transmissometer  was  not  reliable  for  measurement 
comparisons. 

4.2.2.11  Recommendations . 

Results  showed  generally  acceptable  performance  of  the  sensors 
and  system;  however,  to  reduce  uncertainties  due  to  limitations 
noted  above,  additional  Category  Illb  tests  were  recommended  by 
ACW-200B.  Specifically,  test  conditions  which  significantly 
reduce  the  subjectivity  of  measurements  were  advised. 
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5.  SUMMARY  OF  OT&E  TESTING. 


The  RVR  OT&E  period  lasted  approximately  2%  years,  commencing 
with  initial  OT&E  tests  in  March  1992,  and  ending  with  Retest  7 
in  June  1994.  In  addition  to  seven  retests  and  two  specialized 
test  efforts.  Factory,  Software,  and  Design  Qualification  tests 
were  conducted  for  each  design  change. 

A  review  of  the  test  efforts  reveals  that  most  problem  areas  and 
issues  were  discovered  during  initial  OT&E  testing.  Many  of  the 
problems  discovered  in  later  tests  were  related  to  those  found 
initially.  For  example,  system  shutdown  problems  were  noted 
early  in  testing  and  continued  throughout.  Initial  testing 
showed  the  shutdowns  to  be  related  to  water  contamination  on 
sensor  lenses.  When  the  sensors  were  made  more  tolerant  to  water 
on  the  lenses,  it  was  discovered  that  shutdowns  could  be  caused 
by  simulating  precipitation  in  the  sample  volume  during  sunlight 
conditions.  The  involvement  of  sunlight  in  shutdowns  was  not 
understood  prior  to  correction  of  lens  contamination  problems. 

The  possibility  of  snow/ ice  clogging  of  the  VS  presented  an  area 
of  large  concern  for  the  test  team.  Any  buildup  of  snow  or  ice 
in  front  of,  but  not  actually  touching,  the  VS  lens  would  go 
undetected  by  the  window  contamination  sensors.  Transmitted 
light  from  the  sensor  would  then  be  blocked  by  the  snow  or  ice, 
with  the  result  mimicking  a  high  visibility  situation.  Testing 
and  review  of  previously  collected  data  revealed  instances  where 
clogging  could  be  induced  under  weather  conditions  not  uncommon 
in  the  United  States.  Because  tests  with  increased  heater  power 
proved  unsuccessful,  the  look-down  design  was  implemented.  The 
Look-Down  VS  with  conformal  hood  heaters  was  shown  to  be  very 
reliable  under  all  weather  conditions. 

Maintenance,  RMM,  and  documentation  were  areas  that  initially  had 
a  large  number  of  discrepancies.  The  discrepancies  were  a 
concern  to  the  test  team  by  virtue  of  their  shear  number,  rather 
than  the  severity.  As  opposed  to  the  difficulties  with  sensor 
shutdowns,  problems  in  the  maintenance  and  documentation  areas 
became  fewer  as  testing  progressed.  The  exceptions  being  the 
continued  lack  of  an  Off-Site  Maintenance  Manual  and  the  failure 
to  completely  correct  RMS  communication  failures. 

Although  each  test  effort  was  valuable  in  verifying  the 
resolution  or  occurrence  of  problems,  specialized  tests  provided 
the  most  information  in  the  shortest  period  of  time.  Because  of 
their  narrow  focus,  these  tests  provided  a  rapid  method  of 
modifying  the  system  and  verifying  performance.  This  method 
proved  instrumental  in  finally  correcting  the  major  system 
problems  and  obtaining  a  national  deployment  decision. 
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6.  FINAL  RECOMMENDATIONS. 


The  New  Generation  Runway  Visual  Range  (RVR)  system  which  was 
fielded  at  the  completion  of  Operational  Test  and  Evaluation 
(OT&E)  is  considered  by  ACW-200  to  be  a  very  capable  and  reliable 
system.  The  sensors  and  software  differ  drastically  from  those 
initially  tested  in  February  of  1992. 

The  following  specific  recommendations  are  made  in  light  of  the 
final  test  results: 

a.  Additional  testing  should  be  conducted  to  further 
define  system  performance  under  Category  Illb 
conditions.  This  should  include  comparison  testing 
with  the  certified  RVR  systems  currently  in  use  in  the 
United  Kingdom; 

b.  The  use  of  "smarter”  heater  control  algorithms  should 
be  investigated.  This  would  include  determination  of 
conditions  where  heaters  would  not  keep  up  with  heat 
loss,  and  whether  shutting  heaters  off  under  these 
conditions  would  prevent  snow  or  ice  buildup; 

c.  The  RVR  Off-Site  manual  should  be  completed; 

d.  All  system  documentation  should  be  updated  and 
validated; 

e.  The  Remote  Monitoring  Subsystem  (RMS)  communication 
problems  should  be  corrected,  and  the  work-around 
solution  removed  from  the  system  software; 

f.  Automated  Surface  Observing  System  (ASOS)  and  Tower 
Control  Computer  Complex  (TCCC)  interfaces  should  be 
incorporated;  and 

g.  All  other  open  issues  should  be  corrected  as  soon  as 
possible. 

In  light  of  the  lengthy  test  effort  required  for  the  RVR  system, 
the  following  general  recommendations  are  made  for  future  RVR 
modifications  as  well  as  future  procurements: 

a.  Prior  to  developmental  testing,  an  assessment  should  be 
made  to  determine  if  environmental  test  procedures  will 
adequately  determine  system  performance  under  adverse 
weather  conditions.  If,  as  with  the  RVR,  laboratory 
tests  cannot  approximate  real  weather  conditions  that 
might  affect  the  system,  special  testing  should  be 
considered; 
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b.  If  retests  are  required,  a  brief  plan  outlining  goals 
and  procedures  should  be  prepared  for  each  retest. 
Schedule  pressures  should  not  be  allowed  to  circumvent 
this  process; 

c.  Retests  should  not  be  conducted  on  unreleased  software. 
If  it  is  necessary  to  evaluate  engineering  prototypes 
at  the  key  site,  the  test  should  be  conducted  by  the 
system  developers  and  witnessed  by  OT&E  test  members; 

d.  If  initial  OT&E  results  in  discovery  of  significant 
problems,  retest  should  only  be  conducted  when 
developers  have  shown  that  all  discrepancies  have  been 
rectified;  and 

e.  The  test,  team  should  ensure  that  the  system  is  ready 
for  retest  by  witnessing  factory  qualification  and 
verifying  that  a  new  software  version  has  been 
released. 
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7.  ABBREVIATIONS  AND  ACRONYMS. 


ACW 

AFSFO 

ALS 

AOS 

ASOS 

CD 

CRREL 

DVX 

DOT 

DPU 

EU 

FAA 

MCI 

MDT 

MPS 

MRS 

NAS 

OT&E 

PC 

RLIM 

RMM 

RMS 

RVR 

SEA 

SIE 

TCCC 

TRACON 

TSC 

TTR 

YYT 

VS 


Engineering,  Integration,  and  Operational  Evaluation 
Service 

Airway  Facilities  Sector  Field  Office 
Ambient  Light  Sensor 
Operational  Support  Directorate 
Automated  Surface  Observing  System 
Controller  Display 

Cold  Regions  Research  and  Engineering  Laboratory 

Denver  International  Airport 

Department  of  Transportation 

Data  Processing  Unit 

External  User 

Federal  Aviation  Administration 

Kansas  City  International  Airport,  Kansas  City,  MO. 

Maintenance  Data  Terminal 

Maintenance  Processor  Subsystem 

Mountain  Research  Station 

National  Airspace  System 

Operational  Test  &  Evaluation 

Personal  Computer 

Runway  Light  Intensity  Monitor 

Remote  Maintenance  Monitoring 

Remote  Monitoring  Subsystem 

Runway  Visual  Range 

Seattle-Tacoma  Airport,  Seattle,  WA 
Sensor  Interface  Electronics 
Tower  Control  Computer  Complex 
Terminal  Radar  Approach  Control 
Technical  Services  Contract 
Test  Trouble  Report 
St.  John's  Newfoundland 
Visibility  Sensor 
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